-- Logs begin at Wed 2022-05-04 14:05:04 UTC, end at Wed 2022-05-04 15:10:11 UTC. -- May 04 14:36:55.682260 dv-cinder-840493-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 04 14:36:55.696216 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 4 14:36:55 2022] *** May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: nodename: dv-cinder-840493-1-iscsi May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: machine: x86_64 May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: clock source: unix May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: pcre jit disabled May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: detected number of CPU cores: 4 May 04 14:36:55.699476 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: current working directory: / May 04 14:36:55.700107 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: detected binary path: /usr/bin/uwsgi-core May 04 14:36:55.700107 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: your processes number limit is 47680 May 04 14:36:55.700107 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: your memory page size is 4096 bytes May 04 14:36:55.700107 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: detected max file descriptor number: 2048 May 04 14:36:55.700107 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: lock engine: pthread robust mutexes May 04 14:36:55.700107 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: thunder lock: enabled May 04 14:36:55.700107 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 04 14:36:55.700541 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 04 14:36:55.777569 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: Python main interpreter initialized at 0x55c035f639c0 May 04 14:36:55.777569 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: python threads support enabled May 04 14:36:55.777569 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: your server socket listen backlog is limited to 100 connections May 04 14:36:55.777569 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: your mercy for graceful operations on workers is 90 seconds May 04 14:36:55.778117 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: mapped 403077 bytes (393 KB) for 2 cores May 04 14:36:55.778248 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: *** Operational MODE: preforking *** May 04 14:36:55.778769 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: *** uWSGI is running in multiple interpreter mode *** May 04 14:36:55.778769 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: spawned uWSGI master process (pid: 91616) May 04 14:36:55.779015 dv-cinder-840493-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 04 14:36:55.780037 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: spawned uWSGI worker 1 (pid: 91629, cores: 1) May 04 14:36:55.780382 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: spawned uWSGI worker 2 (pid: 91630, cores: 1) May 04 14:36:55.780611 dv-cinder-840493-1-iscsi devstack@keystone.service[91616]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 04 14:37:02.142308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 04 14:37:02.142308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: from cryptography.utils import int_from_bytes May 04 14:37:02.143415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 04 14:37:02.143415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: from cryptography.utils import int_from_bytes May 04 14:37:02.195693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 04 14:37:02.195949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 04 14:37:02.196065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] command line args: [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 04 14:37:02.196390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 04 14:37:02.196570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ================================================================================ {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 04 14:37:02.196854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] admin_token = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.197019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] config_dir = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.197264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.197398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] config_source = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.197627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] debug = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.197766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] list_limit = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] log_config_append = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] log_dir = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.201304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] log_file = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.202991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.203424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.203424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.203661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.204159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.32/identity {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.204500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] publish_errors = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.204934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.205327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.205653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.206052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.206503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.206949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] standard_threads = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.207416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.207874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.208254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.208627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] use_journal = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.209031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] use_json = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.209394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] use_stderr = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.209787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] use_syslog = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.210221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.210689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.211142 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.211618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.212058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.212506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.213001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.213452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.213947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.214400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.214838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.215298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.215727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.216168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.216593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.217075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.217529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.217973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.218432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.218886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.219360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.219793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.220239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.220708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.221207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.221662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.222091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.222512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.222938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.223379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.223817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.224264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.224693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.225168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.225620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.226128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.226573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.227020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.227451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.227878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.228325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.228755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.229260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.229685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.230119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.230549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.231022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.231463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.231894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.232315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.232751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.233242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.233687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.234110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.234549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.234987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.235421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.235861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.236290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.236717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.237210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.237639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.238082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.238519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.238985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.239504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.239935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.240367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.240822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.241294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.241741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.242189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.242640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.243091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.243520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.243958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.244424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.245115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.245568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.246015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.246445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.246888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.247320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.247750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.248176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.248648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.249116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.249582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.250009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.250438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.250865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.251290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.251714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.252150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.252574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.253050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.253490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.253908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.254338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.254785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.255235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.255674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.256260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.256727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.257190 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.257634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] auth.external = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.258065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.258566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.259004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.259438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] auth.password = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.259870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] auth.token = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.260396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.260916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.261364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.261804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.262248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.262691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.263137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.263573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] credential.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.264023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.264462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.264957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.265393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.265865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.266300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.266770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.267227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.267655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.268116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.268614 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.269100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.269564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.269994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.270456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.270889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.271328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.272079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.272517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.272983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.273432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.273861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.274710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.275141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.275600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.276036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.276570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.277047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.277505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.277951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.278384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.278815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.279251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.279671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.280117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.280557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.281039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.281473 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.281932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.282374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.282817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.283255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.283701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.284148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.284596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.285120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.285556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.285984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.286411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.286869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.287290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.287734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.288171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.288603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.289094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.289557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.289999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.290456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.290884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.291340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.291835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.292257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.292691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.293157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.293585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.294033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.294470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.294897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.295326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.295786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.296228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.296669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.297132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.297579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.298022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.298452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.298895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.299354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.299773 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.300211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.300671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.301140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.301604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.302048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.302490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.302918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.303339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.303777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.304218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.304646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.305116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.305571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.306025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.306463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.306928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.307447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.307872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.308328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.308761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.309218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.309753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.310179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.310618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.311067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.311520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.311970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.312411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.312890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.313322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.313751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.314182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.314629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.315088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.315511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.315942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.316374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.316905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.317370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.317819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.318264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.318725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.319154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.319602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.320032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.320470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.320926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] role.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.321385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] role.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.321806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.322244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.322683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.323207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.323668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.324130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.324746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.325233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.325682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.326124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.326564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.327013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.327454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.327892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.328519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.329021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.329469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.329946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.330387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.330832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.331264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.331735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.332182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.332656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.333146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.333571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.334007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.334457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.334929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.335391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.335844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.336278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.336714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.337213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.337666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.338127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.338584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.339040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.339481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.339931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.340371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.340854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.341301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.341729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.342181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.342608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.343068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.343523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.343955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.344391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.344937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.345407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.connection = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.345860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.346311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.346750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.347200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.347630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.347875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.348267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.348519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.348771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.349205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.349462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.349710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.349977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.350222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.350477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.350710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.351104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.351346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.351514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91630) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 04 14:37:02.364063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 04 14:37:02.364063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: from cryptography.utils import int_from_bytes May 04 14:37:02.364919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 04 14:37:02.364919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: from cryptography.utils import int_from_bytes May 04 14:37:02.384403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.384917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.385325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.395071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.412261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 04 14:37:02.412475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 04 14:37:02.412627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] command line args: [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 04 14:37:02.413086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 04 14:37:02.413394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ================================================================================ {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 04 14:37:02.413816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] admin_token = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.414199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] config_dir = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.414517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.414828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] config_source = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.415136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] debug = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.415478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.415881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.416186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.416579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.417050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.417409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] list_limit = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.417733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] log_config_append = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.418034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.418335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] log_dir = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.418634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] log_file = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.419002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.419377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.419762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.420134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.420438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.420737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.421130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.421526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.421901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.422261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.422619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.422978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.423362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.423747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.424153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.424839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.32/identity {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.425149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] publish_errors = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.425524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.425883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.426184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.426555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.426917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.427278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] standard_threads = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.427645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.428013 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.428313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.428610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] use_journal = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.428935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] use_json = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.429230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] use_stderr = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.429338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.429630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.429741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] use_syslog = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.430085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 04 14:37:02.430481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.430845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.431254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.431620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.432005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.432413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.432787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.433173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.433566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.433942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.434325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.434685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.435061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.435434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.435830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.436205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.436570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.436979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.437361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.437737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.438116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.438499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.438897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.439275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.439653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.440009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.440376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.440730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.441107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.441483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.441856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.442229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.442586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.442958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.443328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.443702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.444080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.444449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.444829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.445190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.445545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.445899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.446252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.446624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.446984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.447358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.447714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.448147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.448500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.448922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.449286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.449662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.450020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.450381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.450884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.451268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.451634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.452000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.452357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.452735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.453129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.453519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.453878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.454269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.454645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.455004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.455371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.455745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.456145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.456512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.456912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.457289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.457667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.458041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.458406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.458780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.459147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.459514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.459885 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.460260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.460623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.461010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.461374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.461737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.462128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.462519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.462888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.463265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.463682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.464047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.464411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.464788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.465197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.465554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.465912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.466273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.466629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.467014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.467376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.467752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.468117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.468508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.468939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.469323 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.469678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] auth.external = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.470034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.470457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.470816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.471171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] auth.password = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.471542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] auth.token = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.471907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.472268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.472641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.473049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.473422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.473814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.474172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.474540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] credential.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.474911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.475287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.475669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.476122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.476501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.476900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.477292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.477675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.478037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.478435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.478854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.479299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.479690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.480058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.480444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.480878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.481255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.481645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.482018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.482376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.482750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.483131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.483530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.483894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.484286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.484651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.485084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.485447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.485823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.486197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.486560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.486922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.487318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.487676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.488041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.488419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.488841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.489200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.489574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.489933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.490301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.490675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.491050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.491429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.491816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.492213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.492576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.492984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.493341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.493703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.494075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.494435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.494845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.495213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.495588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.495953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.496336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.496708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.497102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.497475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.497847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.498214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.498321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.498680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.499043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.499405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.499766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.500145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.500517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.500914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.501294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.501670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.502025 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.502380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.502770 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.503141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.503502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.503861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.504221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.504592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.504996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.505382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.505740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.506110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.506483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.506856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.507234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.507599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.507958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.508327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.508690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.509080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.509474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.509851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.510223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.510669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.511043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.511399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.511787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.512160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.512520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.512943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.513309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.513681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.514078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.514452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.514833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.515189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.515559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.515920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.516290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.516652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.517075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.517445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.517801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.518177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.518561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.518944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.519318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.519679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.520056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.520446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.520825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.521196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.521863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.522229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.522588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] role.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.522978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] role.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.523334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.523708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.524083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.524467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.524865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.525259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.525634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.526018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.526437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.526796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.527168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.527560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.527936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.528309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.528916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.529280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.529657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.530053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.530429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.530805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.531159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.531541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.531919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.532332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.533629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.534057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.534454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.534874 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.535297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.535729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.536130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.536526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.536943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.537341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.537738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.538165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.538562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.538982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.539388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.539784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.540185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.540596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.541047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.541445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.541914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.542310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.542715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.543139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.543527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.543927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.544394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.544857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.connection = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.545294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.545811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.546220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.546621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.547027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.547428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.547819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.548216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.548634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.549083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.549491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.549889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.550300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.550696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.551130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.551518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.551917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.552309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 04 14:37:02.552632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 04 14:37:02.558590 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.558819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.580548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.580890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.581184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.590602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.610103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.610425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.630298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.630537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.630839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.631007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.631129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.631379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.651047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.656305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.656616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.656934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.657067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.657188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.657457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.657611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.669943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.685333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.685576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.685878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.686020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.686168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.686406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.686627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91630) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.689787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.690106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.704483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.704922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 04 14:37:02.705157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.705377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 04 14:37:02.705594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.705814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.706057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.706275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.719954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.720356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.720583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.720840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.731483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.735812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.736224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.736616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.736894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.737154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.737382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.737619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.737619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.738028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.738262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.738262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.738666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.738666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.738666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.739271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.739479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.739683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.753300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.753713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.753713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.754180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.754180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.754603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.754603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.762841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.765857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.766125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.766351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.766596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.766818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.767066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.767066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 04 14:37:02.768845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.770560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.770849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.776703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.777061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.777286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.778346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.778578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 04 14:37:02.778875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.779126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 04 14:37:02.779347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.779592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.779848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.780111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.784706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.785034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.789708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.789975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.790264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.790551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.790809 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.791079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.791312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.791536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.791756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.791990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.792531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.792531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.800977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.804293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.804581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.804857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.805138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.805413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.805786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.806056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.806304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.806580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.806844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.807128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.810549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.810854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.815422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.819106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.819416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.824538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.828199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.832025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.832334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.832605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.832986 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.833241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.833458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.833715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.835382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.835674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.841088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.841372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.841638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.848985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.849340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.852673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.853106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.853379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.853645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.853867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.854164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.854471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.854758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.855036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.855344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.855561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.855865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.861724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.863481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.863763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.866861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.867158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.867445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.867727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.868005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.874758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.875108 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.879632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.881684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.881983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.882274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.882512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.882797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.883074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.883343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.883593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.883814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.888699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.892311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.896090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.896371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.897001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.897282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.897556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.897783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.902093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.905361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.908710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.910375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.912057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.912324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.912618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.917968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.918379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.918638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.918914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.920214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.921896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.925158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.925460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.925761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.926047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.926993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.928738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.929021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.932091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.932444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.932741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.933150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.933385 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.935447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.935755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.936037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.944362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91630) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.944750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.945023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.945250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.945566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.945821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.946060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.946280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.946541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.946769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.946993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.947234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.947451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.947668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.947882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.948112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.948334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.948568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.948780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91630) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.949052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.949052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.967597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.970903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.974340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.975943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.977594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.977848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.978132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.985643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.986939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55c035f639c0 pid: 91630 (default app) May 04 14:37:02.987241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:02.990485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.990756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a64dae-152f-4c26-a6d9-e2879dff7a41 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:02.991011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.991238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a64dae-152f-4c26-a6d9-e2879dff7a41 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:02.991477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.991772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a64dae-152f-4c26-a6d9-e2879dff7a41 None None] PATH_INFO: `/v3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:02.992051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:02.998850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1/1] 10.222.0.32 () {50 vars in 791 bytes} [Wed May 4 14:37:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:03.000629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:03.001016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.001213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.009430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 04 14:37:03.009720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.010001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.010270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.010596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.010870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.011136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.011397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.011668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.011940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.012212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.012473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.012729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.013897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 04 14:37:03.052093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55c035f639c0 pid: 91629 (default app) May 04 14:37:12.015082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944857f-3460-4889-bb17-0f7f8f1e346d None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:12.015929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944857f-3460-4889-bb17-0f7f8f1e346d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:12.016052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944857f-3460-4889-bb17-0f7f8f1e346d None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:12.017939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 2/2] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:12.026963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:12.027737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:12.027908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:12.062915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG oslo_db.sqlalchemy.engines [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 04 14:37:12.268559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG passlib.handlers.bcrypt [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91629) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 04 14:37:12.269056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG passlib.handlers.bcrypt [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91629) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 04 14:37:12.337724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:12.340649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:12.498903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1/3] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 04 14:37:12.740178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944857f-3460-4889-bb17-0f7f8f1e346d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:12.780880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG oslo_db.sqlalchemy.engines [None req-2944857f-3460-4889-bb17-0f7f8f1e346d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91630) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 04 14:37:12.914700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944857f-3460-4889-bb17-0f7f8f1e346d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:12.920551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15401a-44ab-4c05-b956-34bbc7265fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:12.921502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15401a-44ab-4c05-b956-34bbc7265fdc admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:12.921923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15401a-44ab-4c05-b956-34bbc7265fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:12.922261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15401a-44ab-4c05-b956-34bbc7265fdc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:12.931948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15401a-44ab-4c05-b956-34bbc7265fdc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:12.986168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2a15401a-44ab-4c05-b956-34bbc7265fdc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 04 14:37:12.986504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15401a-44ab-4c05-b956-34bbc7265fdc admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:12.997953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 3/4] 10.222.0.32 () {58 vars in 1324 bytes} [Wed May 4 14:37:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:37:13.004503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:13.020607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9972-e614-4cbb-b208-a7d14d739463 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:13.025895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0538d-413f-45ba-ad4c-553febcbf0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:13.027178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0538d-413f-45ba-ad4c-553febcbf0dd admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:13.027991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0538d-413f-45ba-ad4c-553febcbf0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:13.027991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0538d-413f-45ba-ad4c-553febcbf0dd admin admin] PATH_INFO: `/v3/projects/62c598d6329d4180a1bbd38c5500b8cc` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:13.029827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0538d-413f-45ba-ad4c-553febcbf0dd admin admin] RBAC: Authorizing `identity:get_project(project_id=62c598d6329d4180a1bbd38c5500b8cc)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:13.083680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79e0538d-413f-45ba-ad4c-553febcbf0dd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 04 14:37:13.084082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0538d-413f-45ba-ad4c-553febcbf0dd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:13.089454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 2/5] 10.222.0.32 () {56 vars in 1207 bytes} [Wed May 4 14:37:13 2022] GET /identity/v3/projects/62c598d6329d4180a1bbd38c5500b8cc => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:15.398794 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd962-5d46-48e9-9ec7-5481245fafe4 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:15.399354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd962-5d46-48e9-9ec7-5481245fafe4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:15.399794 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd962-5d46-48e9-9ec7-5481245fafe4 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:15.408554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 4/6] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:15.415985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb939e-8e4d-498b-b8c3-ad63563a27c6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:15.416487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb939e-8e4d-498b-b8c3-ad63563a27c6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:15.417186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb939e-8e4d-498b-b8c3-ad63563a27c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:15.448235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-16cb939e-8e4d-498b-b8c3-ad63563a27c6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:15.455639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-16cb939e-8e4d-498b-b8c3-ad63563a27c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:15.469920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 3/7] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 04 14:37:15.493295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd962-5d46-48e9-9ec7-5481245fafe4 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:15.505807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd962-5d46-48e9-9ec7-5481245fafe4 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:15.510197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df015be-faf7-45c1-9f65-97383e7fec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:15.510913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df015be-faf7-45c1-9f65-97383e7fec0b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:15.511226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df015be-faf7-45c1-9f65-97383e7fec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:15.511500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df015be-faf7-45c1-9f65-97383e7fec0b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:15.520126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df015be-faf7-45c1-9f65-97383e7fec0b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:15.522362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df015be-faf7-45c1-9f65-97383e7fec0b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:15.532703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 5/8] 10.222.0.32 () {58 vars in 1324 bytes} [Wed May 4 14:37:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 04 14:37:15.541353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb939e-8e4d-498b-b8c3-ad63563a27c6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:15.551160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb939e-8e4d-498b-b8c3-ad63563a27c6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:15.555678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb4189-5f80-47c2-9f25-3318075e8764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:15.556578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb4189-5f80-47c2-9f25-3318075e8764 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:15.556883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb4189-5f80-47c2-9f25-3318075e8764 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:15.557158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb4189-5f80-47c2-9f25-3318075e8764 admin admin] PATH_INFO: `/v3/users/f6614ed1ab4445508051598d04c940bf` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:15.558729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb4189-5f80-47c2-9f25-3318075e8764 admin admin] RBAC: Authorizing `identity:get_user(user_id=f6614ed1ab4445508051598d04c940bf)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:15.560576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb4189-5f80-47c2-9f25-3318075e8764 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:15.562275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 4/9] 10.222.0.32 () {56 vars in 1198 bytes} [Wed May 4 14:37:15 2022] GET /identity/v3/users/f6614ed1ab4445508051598d04c940bf => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:17.957320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804133f3-6057-473f-a3d6-91fc585e07e2 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:17.958127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804133f3-6057-473f-a3d6-91fc585e07e2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:17.958345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804133f3-6057-473f-a3d6-91fc585e07e2 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:17.960854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 6/10] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:17.969306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a653-9f22-45b6-9c2d-f0d2897bd224 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:17.970075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a653-9f22-45b6-9c2d-f0d2897bd224 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:17.970216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a653-9f22-45b6-9c2d-f0d2897bd224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:17.989028 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3223a653-9f22-45b6-9c2d-f0d2897bd224 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:17.990434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3223a653-9f22-45b6-9c2d-f0d2897bd224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:18.023073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6bab-bafe-4fb6-bba3-72a686b7c0ef None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.023362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6bab-bafe-4fb6-bba3-72a686b7c0ef None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.023627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6bab-bafe-4fb6-bba3-72a686b7c0ef None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.025031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 7/11] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:18.033585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.034017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.034410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.077886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG passlib.handlers.bcrypt [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91630) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 04 14:37:18.078462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG passlib.handlers.bcrypt [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91630) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 04 14:37:18.085484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 5/12] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:18.108752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223a653-9f22-45b6-9c2d-f0d2897bd224 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:18.125728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:18.127377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:18.242065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223a653-9f22-45b6-9c2d-f0d2897bd224 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:18.268147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 8/13] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:37:18.278473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:18.279292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.279572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.279848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.292174 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:18.303262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:18.303970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:18.305737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:18.307207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 6/14] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:18.314702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:18.430813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7ede0-a6a1-47a4-8713-c699f301187b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:18.439771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae83f3-af4e-478c-b83f-b1c6f6287edc None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:18.467033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:18.467719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.468002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.468277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.469072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:18.469558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:18.470993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:18.475362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:18.476306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.476592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.476903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.483876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 7/15] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:18.489829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:18.547681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:18.549267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:18.598432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a830c-1e7e-4571-9150-4c89006467c8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:18.637011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:18.637928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.638216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.638499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] PATH_INFO: `/v3/users/f6614ed1ab4445508051598d04c940bf` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.639866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=f6614ed1ab4445508051598d04c940bf)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:18.641447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:18.643264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 8/16] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:18 2022] GET /identity/v3/users/f6614ed1ab4445508051598d04c940bf => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:18.649636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:18.660406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 9/17] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:37:18.666576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:18.761517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a191f0-73e2-40cb-830b-83890b1d2fc6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:18.785968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f22aa-2d81-4468-8c79-925c040d4f31 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:18.813724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:18.814472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.814765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.815047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.816448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:18.818042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:18.819868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 9/18] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:18.826349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:18.830209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:18.831566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:18.832035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:18.832376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:18.970631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:18.972148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:18.973281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df79b97-b2a2-46b7-9876-d5656a2ebb17 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:37:18.994323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:18.995922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:18.998157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:37:18.998157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:37:18.998157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:37:18.998157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:18.998157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 10/19] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:19.004573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:19.013924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d0b33-d739-4a40-a602-eeb8a2b2c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:19.014605 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d0b33-d739-4a40-a602-eeb8a2b2c95e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:19.014886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d0b33-d739-4a40-a602-eeb8a2b2c95e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:19.015159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d0b33-d739-4a40-a602-eeb8a2b2c95e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:19.016007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d0b33-d739-4a40-a602-eeb8a2b2c95e None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.domain.id=default, user.id=f6614ed1ab4445508051598d04c940bf) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:19.016318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d0b33-d739-4a40-a602-eeb8a2b2c95e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:19.017838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d0b33-d739-4a40-a602-eeb8a2b2c95e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:19.025926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 10/20] 10.222.0.32 () {56 vars in 1322 bytes} [Wed May 4 14:37:18 2022] GET /identity/v3/role_assignments?user.id=f6614ed1ab4445508051598d04c940bf&scope.domain.id=default&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 233 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:19.147388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c26d-4d1a-4f3d-a7f7-7a39c6c10401 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:19.185972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec931439-6c3f-49f3-ac26-d127fda1d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:19.186688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec931439-6c3f-49f3-ac26-d127fda1d243 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:19.186973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec931439-6c3f-49f3-ac26-d127fda1d243 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:19.187251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec931439-6c3f-49f3-ac26-d127fda1d243 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:19.188011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec931439-6c3f-49f3-ac26-d127fda1d243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:19.188325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec931439-6c3f-49f3-ac26-d127fda1d243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:19.189815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec931439-6c3f-49f3-ac26-d127fda1d243 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:19.210141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 11/21] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:21.639025 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb887-3253-405d-91d1-44a37f1505db None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:21.639512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb887-3253-405d-91d1-44a37f1505db None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:21.639512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbb887-3253-405d-91d1-44a37f1505db None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:21.641198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 11/22] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:21.646839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e8f4-37d9-4ada-845e-5d685125b178 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:21.648137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e8f4-37d9-4ada-845e-5d685125b178 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:21.648615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e8f4-37d9-4ada-845e-5d685125b178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:21.668039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-3154e8f4-37d9-4ada-845e-5d685125b178 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:21.669659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-3154e8f4-37d9-4ada-845e-5d685125b178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:21.740231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 12/23] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:21.762894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbb887-3253-405d-91d1-44a37f1505db None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:21.879768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbb887-3253-405d-91d1-44a37f1505db None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:21.920223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:21.921436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:21.921933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:21.922431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:21.934522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:21.942631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:21.943543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:21.944322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:21.947118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 12/24] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:21.953546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e8f4-37d9-4ada-845e-5d685125b178 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:22.056736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e8f4-37d9-4ada-845e-5d685125b178 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:22.092249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:22.093393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:22.093878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:22.094348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:22.095319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:22.095825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:22.097490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:22.111429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 13/25] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:22.117445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:22.224823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610190b-00cb-43d9-8a85-e565f9f4b05a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:22.263239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:22.264536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:22.265066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:22.265538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] PATH_INFO: `/v3/users/f6614ed1ab4445508051598d04c940bf` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:22.267100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] RBAC: Authorizing `identity:get_user(user_id=f6614ed1ab4445508051598d04c940bf)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:22.268860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:22.271151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 13/26] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:22 2022] GET /identity/v3/users/f6614ed1ab4445508051598d04c940bf => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:22.277563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:22.388937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d818af2-9eaf-469c-8199-fb5f2269fd97 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:22.425246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d418e6-468a-43ce-8f2c-9e5d3d1c16d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:22.426067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d418e6-468a-43ce-8f2c-9e5d3d1c16d4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:22.426359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d418e6-468a-43ce-8f2c-9e5d3d1c16d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:22.426637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d418e6-468a-43ce-8f2c-9e5d3d1c16d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:22.428184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d418e6-468a-43ce-8f2c-9e5d3d1c16d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:22.429727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d418e6-468a-43ce-8f2c-9e5d3d1c16d4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:22.431930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 14/27] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:22.437799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:22.673042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9881b-86fe-4ed4-9acf-18be528b5ffb None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:22.711740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839699a-9a9d-4e1d-b957-9f23d898ebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:22.712653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839699a-9a9d-4e1d-b957-9f23d898ebc1 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:22.712979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839699a-9a9d-4e1d-b957-9f23d898ebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:22.713265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839699a-9a9d-4e1d-b957-9f23d898ebc1 None admin] PATH_INFO: `/v3/domains/default/users/f6614ed1ab4445508051598d04c940bf/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:22.715538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839699a-9a9d-4e1d-b957-9f23d898ebc1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f6614ed1ab4445508051598d04c940bf, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:22.717135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839699a-9a9d-4e1d-b957-9f23d898ebc1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:22.864429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 14/28] 10.222.0.32 () {58 vars in 1361 bytes} [Wed May 4 14:37:22 2022] PUT /identity/v3/domains/default/users/f6614ed1ab4445508051598d04c940bf/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:22.917347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b54b3c-d29b-460b-a334-bcae9b92ecc2 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:22.918355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b54b3c-d29b-460b-a334-bcae9b92ecc2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:22.918832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b54b3c-d29b-460b-a334-bcae9b92ecc2 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:22.920907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 15/29] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:22.927203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7de56-bd90-4f0a-afbe-2227d340fe30 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:22.927489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7de56-bd90-4f0a-afbe-2227d340fe30 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:22.927702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7de56-bd90-4f0a-afbe-2227d340fe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:22.953736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-77b7de56-bd90-4f0a-afbe-2227d340fe30 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:22.955353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-77b7de56-bd90-4f0a-afbe-2227d340fe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:23.054967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 15/30] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:37:23.086791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b54b3c-d29b-460b-a334-bcae9b92ecc2 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:23.307270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b54b3c-d29b-460b-a334-bcae9b92ecc2 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:23.371502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6b38a-fda2-4ae4-b2b9-51148ca9bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:23.372540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6b38a-fda2-4ae4-b2b9-51148ca9bf36 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:23.372992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6b38a-fda2-4ae4-b2b9-51148ca9bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:23.373404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6b38a-fda2-4ae4-b2b9-51148ca9bf36 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:23.374639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6b38a-fda2-4ae4-b2b9-51148ca9bf36 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:23.377047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6b38a-fda2-4ae4-b2b9-51148ca9bf36 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:23.441062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714dc45-e840-465d-84e2-70e8b9e21dd8 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:23.441716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714dc45-e840-465d-84e2-70e8b9e21dd8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:23.442222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714dc45-e840-465d-84e2-70e8b9e21dd8 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:23.456829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 16/31] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:23.458733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d790d-c423-4aa5-a4dc-84a3ec0d7a53 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:23.459284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d790d-c423-4aa5-a4dc-84a3ec0d7a53 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:23.459784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d790d-c423-4aa5-a4dc-84a3ec0d7a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:23.498054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5f3d790d-c423-4aa5-a4dc-84a3ec0d7a53 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:23.500437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5f3d790d-c423-4aa5-a4dc-84a3ec0d7a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:23.562427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 16/32] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 4 14:37:23 2022] POST /identity/v3/roles => generated 219 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:23.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0093-9293-474f-8eb4-8eaf4a506844 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:23.564862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0093-9293-474f-8eb4-8eaf4a506844 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:23.565351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0093-9293-474f-8eb4-8eaf4a506844 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:23.567242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 17/33] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:23.577192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526941-81ea-4cfa-87e0-cdd1cf172e49 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:23.577843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526941-81ea-4cfa-87e0-cdd1cf172e49 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:23.578320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7526941-81ea-4cfa-87e0-cdd1cf172e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:23.608471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a7526941-81ea-4cfa-87e0-cdd1cf172e49 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:23.610561 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a7526941-81ea-4cfa-87e0-cdd1cf172e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:23.671743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 17/34] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:23.707056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d790d-c423-4aa5-a4dc-84a3ec0d7a53 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:23.722640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 18/35] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:37:23.748119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7526941-81ea-4cfa-87e0-cdd1cf172e49 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:23.862578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d790d-c423-4aa5-a4dc-84a3ec0d7a53 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:23.902340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:23.903400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:23.903832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:23.904311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:23.905342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:23.907122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:23.928701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 18/36] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 4 14:37:23 2022] POST /identity/v3/roles => generated 225 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:23.967718 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7526941-81ea-4cfa-87e0-cdd1cf172e49 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:24.014207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11755e-aa38-4edd-94e5-a2c38ac8afc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:24.015412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11755e-aa38-4edd-94e5-a2c38ac8afc5 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.015848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11755e-aa38-4edd-94e5-a2c38ac8afc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.016277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11755e-aa38-4edd-94e5-a2c38ac8afc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.030238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11755e-aa38-4edd-94e5-a2c38ac8afc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:24.032347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11755e-aa38-4edd-94e5-a2c38ac8afc5 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-2a11755e-aa38-4edd-94e5-a2c38ac8afc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:37:24.053043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:24.054106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:24.055135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:37:24.055135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:37:24.055135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:37:24.055135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:24.056903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 19/37] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:24.064976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:24.206915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff417c-47ce-4fc5-a744-700e0dcacf1b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:24.220957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2383-f283-4700-996e-2085dfe437f7 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.221622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2383-f283-4700-996e-2085dfe437f7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.222124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2383-f283-4700-996e-2085dfe437f7 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.223827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 20/38] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:24.234437 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc601b-aa88-4092-86c5-7649018b7ac3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.235060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc601b-aa88-4092-86c5-7649018b7ac3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.235537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc601b-aa88-4092-86c5-7649018b7ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.261075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-dfcc601b-aa88-4092-86c5-7649018b7ac3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:24.262763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-dfcc601b-aa88-4092-86c5-7649018b7ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:24.275695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:24.277033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.277496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.278005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.279464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:24.280024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:24.283420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:24.337192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 19/39] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:24.344364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:24.346345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 21/40] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:24.369840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc601b-aa88-4092-86c5-7649018b7ac3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:24.484652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc601b-aa88-4092-86c5-7649018b7ac3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:24.525066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5868c84-3487-45d2-b692-ea07bef213a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:24.525765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5868c84-3487-45d2-b692-ea07bef213a1 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.526085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5868c84-3487-45d2-b692-ea07bef213a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.526454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5868c84-3487-45d2-b692-ea07bef213a1 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.527254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5868c84-3487-45d2-b692-ea07bef213a1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:24.528719 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5868c84-3487-45d2-b692-ea07bef213a1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:24.536755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e07d2c-d12e-4fd7-b7eb-ee57a0351c7a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:24.546043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 22/41] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 4 14:37:24 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:24.613264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42005ff9-333e-4ee8-b270-61d396a89751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:24.614956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005ff9-333e-4ee8-b270-61d396a89751 None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.615755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005ff9-333e-4ee8-b270-61d396a89751 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.616511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005ff9-333e-4ee8-b270-61d396a89751 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.617830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42005ff9-333e-4ee8-b270-61d396a89751 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:24.621261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42005ff9-333e-4ee8-b270-61d396a89751 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:24.677706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 20/42] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:37:24 2022] POST /identity/v3/projects => generated 301 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:24.788543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19181c67-0c4d-45d9-8641-d71330f7248e None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.788940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19181c67-0c4d-45d9-8641-d71330f7248e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.789162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19181c67-0c4d-45d9-8641-d71330f7248e None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.790700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 23/43] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:24.798466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5058e1-e5a6-48a6-bef1-9796396d029c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:24.799294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5058e1-e5a6-48a6-bef1-9796396d029c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:24.799475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5058e1-e5a6-48a6-bef1-9796396d029c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:24.823785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7d5058e1-e5a6-48a6-bef1-9796396d029c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:24.825071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7d5058e1-e5a6-48a6-bef1-9796396d029c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:24.917606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 21/44] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:24.941559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19181c67-0c4d-45d9-8641-d71330f7248e None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:25.065415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19181c67-0c4d-45d9-8641-d71330f7248e None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:25.100264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b20b1-083d-40ff-a791-d23be19b850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:25.101120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b20b1-083d-40ff-a791-d23be19b850b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:25.101405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b20b1-083d-40ff-a791-d23be19b850b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:25.101682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b20b1-083d-40ff-a791-d23be19b850b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:25.102945 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b20b1-083d-40ff-a791-d23be19b850b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:25.104522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b20b1-083d-40ff-a791-d23be19b850b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:25.106248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 24/45] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:25.112114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5058e1-e5a6-48a6-bef1-9796396d029c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:25.218039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5058e1-e5a6-48a6-bef1-9796396d029c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:25.261272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0bd3e8-e202-437f-8792-46ef20135f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:25.262042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bd3e8-e202-437f-8792-46ef20135f1f None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:25.262334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bd3e8-e202-437f-8792-46ef20135f1f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:25.262610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0bd3e8-e202-437f-8792-46ef20135f1f None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:25.263357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bd3e8-e202-437f-8792-46ef20135f1f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:25.264916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0bd3e8-e202-437f-8792-46ef20135f1f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:25.294098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 22/46] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:37:25 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:25.801598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788016f-aa61-4153-b2ae-1b2a818d2ef5 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:25.801854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788016f-aa61-4153-b2ae-1b2a818d2ef5 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:25.802157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788016f-aa61-4153-b2ae-1b2a818d2ef5 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:25.803748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 25/47] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:25.809477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b15d4-41cd-4a8f-af19-9622e6da747c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:25.809967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b15d4-41cd-4a8f-af19-9622e6da747c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:25.810142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b15d4-41cd-4a8f-af19-9622e6da747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:25.829499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3e4b15d4-41cd-4a8f-af19-9622e6da747c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:25.830711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3e4b15d4-41cd-4a8f-af19-9622e6da747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:25.901575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 23/48] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:25.925442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788016f-aa61-4153-b2ae-1b2a818d2ef5 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:26.032079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788016f-aa61-4153-b2ae-1b2a818d2ef5 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:26.069573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:26.070654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:26.071115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:26.071572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:26.085927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:26.094993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:26.096893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:26.097760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:26.101098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 26/49] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:26.104234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b15d4-41cd-4a8f-af19-9622e6da747c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:26.207503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b15d4-41cd-4a8f-af19-9622e6da747c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:26.243491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:26.244188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:26.244468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:26.244750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:26.245622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:26.245921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:26.247358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:26.256446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 24/50] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:26.262481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:26.369772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e161-301a-4462-a3e1-7ce07dd3c45f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:26.423409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:26.424456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:26.424863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:26.425237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] PATH_INFO: `/v3/users/f6614ed1ab4445508051598d04c940bf` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:26.426898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] RBAC: Authorizing `identity:get_user(user_id=f6614ed1ab4445508051598d04c940bf)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:26.428496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:26.430249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 27/51] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:26 2022] GET /identity/v3/users/f6614ed1ab4445508051598d04c940bf => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:26.436955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:26.616219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f666f5-12ff-4396-94c2-9f14214e6c05 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:26.664512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb7162-2038-46dd-b203-9413f2c7302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:26.665556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb7162-2038-46dd-b203-9413f2c7302c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:26.665946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb7162-2038-46dd-b203-9413f2c7302c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:26.666326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb7162-2038-46dd-b203-9413f2c7302c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:26.667955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb7162-2038-46dd-b203-9413f2c7302c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:26.669477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb7162-2038-46dd-b203-9413f2c7302c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:26.671228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 25/52] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:26.680384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:26.791987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f4069-5501-4eb0-a54b-8d92118fb302 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:26.831542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9bb0d-7f58-4104-8e93-004878fb8984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:26.832213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9bb0d-7f58-4104-8e93-004878fb8984 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:26.832493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9bb0d-7f58-4104-8e93-004878fb8984 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:26.832766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9bb0d-7f58-4104-8e93-004878fb8984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:26.833645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9bb0d-7f58-4104-8e93-004878fb8984 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.domain.id=default, user.id=f6614ed1ab4445508051598d04c940bf) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:26.833953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9bb0d-7f58-4104-8e93-004878fb8984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:26.835430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9bb0d-7f58-4104-8e93-004878fb8984 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:26.845824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 28/53] 10.222.0.32 () {56 vars in 1322 bytes} [Wed May 4 14:37:26 2022] GET /identity/v3/role_assignments?user.id=f6614ed1ab4445508051598d04c940bf&scope.domain.id=default&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 530 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:27.633568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55140dc-e229-480a-9e3e-480950f64f00 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:27.634086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55140dc-e229-480a-9e3e-480950f64f00 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:27.634086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55140dc-e229-480a-9e3e-480950f64f00 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:27.635444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 26/54] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:27.641412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee48b4-988c-4548-be77-da844fa3f54d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:27.642199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee48b4-988c-4548-be77-da844fa3f54d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:27.642199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee48b4-988c-4548-be77-da844fa3f54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:27.661423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-14ee48b4-988c-4548-be77-da844fa3f54d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:27.662655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-14ee48b4-988c-4548-be77-da844fa3f54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:27.740930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 29/55] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:27.765121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55140dc-e229-480a-9e3e-480950f64f00 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:27.896416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55140dc-e229-480a-9e3e-480950f64f00 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:27.935059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304ac3c-1520-495c-96d3-f8d648c72a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:27.935902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304ac3c-1520-495c-96d3-f8d648c72a3b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:27.936192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304ac3c-1520-495c-96d3-f8d648c72a3b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:27.936478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304ac3c-1520-495c-96d3-f8d648c72a3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:27.937892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304ac3c-1520-495c-96d3-f8d648c72a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:27.939424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304ac3c-1520-495c-96d3-f8d648c72a3b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:27.941272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 27/56] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:27.950501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee48b4-988c-4548-be77-da844fa3f54d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:28.062665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee48b4-988c-4548-be77-da844fa3f54d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:28.103374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fedc4-8611-4075-a0ec-f2b550c7f83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:28.104073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fedc4-8611-4075-a0ec-f2b550c7f83f None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:28.104460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fedc4-8611-4075-a0ec-f2b550c7f83f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:28.104735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fedc4-8611-4075-a0ec-f2b550c7f83f None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:28.105515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fedc4-8611-4075-a0ec-f2b550c7f83f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:28.107050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fedc4-8611-4075-a0ec-f2b550c7f83f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:28.140451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 30/57] 10.222.0.32 () {60 vars in 1140 bytes} [Wed May 4 14:37:27 2022] POST /identity/v3/projects => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:30.462796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646db60-78fa-42d4-ba77-0a885e6a052f None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:30.463328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646db60-78fa-42d4-ba77-0a885e6a052f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:30.463328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646db60-78fa-42d4-ba77-0a885e6a052f None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:30.464888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 28/58] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:30.470627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b24b7-8212-422c-bed5-b23e07faed31 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:30.471125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b24b7-8212-422c-bed5-b23e07faed31 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:30.471208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b24b7-8212-422c-bed5-b23e07faed31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:30.488713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4a6b24b7-8212-422c-bed5-b23e07faed31 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:30.489988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4a6b24b7-8212-422c-bed5-b23e07faed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:30.562390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 31/59] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:30.587439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646db60-78fa-42d4-ba77-0a885e6a052f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:30.703801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646db60-78fa-42d4-ba77-0a885e6a052f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:30.739287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db0080-999c-42ae-8a4e-11f6dfe51adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:30.740088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0080-999c-42ae-8a4e-11f6dfe51adf None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:30.740377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0080-999c-42ae-8a4e-11f6dfe51adf None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:30.740656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db0080-999c-42ae-8a4e-11f6dfe51adf None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:30.741915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0080-999c-42ae-8a4e-11f6dfe51adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:30.743387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db0080-999c-42ae-8a4e-11f6dfe51adf None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:30.745220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 29/60] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:30.751179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b24b7-8212-422c-bed5-b23e07faed31 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:30.852292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b24b7-8212-422c-bed5-b23e07faed31 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:30.891715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d930d-4fe9-4f0d-88a7-369c66ff06ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:30.892537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d930d-4fe9-4f0d-88a7-369c66ff06ae None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:30.892846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d930d-4fe9-4f0d-88a7-369c66ff06ae None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:30.893262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d930d-4fe9-4f0d-88a7-369c66ff06ae None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:30.894188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d930d-4fe9-4f0d-88a7-369c66ff06ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:30.895843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d930d-4fe9-4f0d-88a7-369c66ff06ae None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:30.924234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 32/61] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:37:30 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:37:34.720570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38b772-52a0-4fe8-bbb6-4d48f9596e5f None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:34.720570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38b772-52a0-4fe8-bbb6-4d48f9596e5f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:34.720570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38b772-52a0-4fe8-bbb6-4d48f9596e5f None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:34.720570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 30/62] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:34.728468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f11f-4118-4f09-b9f5-49ea8b314e32 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:34.729563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f11f-4118-4f09-b9f5-49ea8b314e32 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:34.730060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9f11f-4118-4f09-b9f5-49ea8b314e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:34.750724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a6e9f11f-4118-4f09-b9f5-49ea8b314e32 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:34.752307 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a6e9f11f-4118-4f09-b9f5-49ea8b314e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:34.823883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 33/63] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:34.899347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38b772-52a0-4fe8-bbb6-4d48f9596e5f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.046848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38b772-52a0-4fe8-bbb6-4d48f9596e5f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.102732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.103949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.104482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.105017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.115092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855b87c-a777-4eab-b8b5-0b6b63ac43dd None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.115745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855b87c-a777-4eab-b8b5-0b6b63ac43dd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.116235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855b87c-a777-4eab-b8b5-0b6b63ac43dd None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.118170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 34/64] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:35.124216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e7197-d364-4384-91b4-9dd4abce9ac0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.124846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e7197-d364-4384-91b4-9dd4abce9ac0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.125367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e7197-d364-4384-91b4-9dd4abce9ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.125873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:35.133643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:35.134635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:35.135589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 31/65] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:35.145874 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.148807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b58e7197-d364-4384-91b4-9dd4abce9ac0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:35.150350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b58e7197-d364-4384-91b4-9dd4abce9ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:35.225563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 35/66] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:37:35.250358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5defd3-fc7d-4278-9928-ab9fa6a6e94d None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.251074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5defd3-fc7d-4278-9928-ab9fa6a6e94d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.251567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5defd3-fc7d-4278-9928-ab9fa6a6e94d None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.253783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 36/67] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:35.254859 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5defd3-fc7d-4278-9928-ab9fa6a6e94d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.304456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e884d-d8f9-425b-95fc-9347e205e4f0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.352069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0627168-9aa4-407e-922a-6aacd188491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.352845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627168-9aa4-407e-922a-6aacd188491c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.353321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627168-9aa4-407e-922a-6aacd188491c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.353742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0627168-9aa4-407e-922a-6aacd188491c None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.355726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0627168-9aa4-407e-922a-6aacd188491c None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:35.356235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0627168-9aa4-407e-922a-6aacd188491c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:35.357881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0627168-9aa4-407e-922a-6aacd188491c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:35.366423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5defd3-fc7d-4278-9928-ab9fa6a6e94d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.367640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 32/68] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:35.369453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36e5a0-3417-4373-8d49-a7c34e0fdb35 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.370017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36e5a0-3417-4373-8d49-a7c34e0fdb35 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.370540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36e5a0-3417-4373-8d49-a7c34e0fdb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.389719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-8d36e5a0-3417-4373-8d49-a7c34e0fdb35 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:35.391408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8d36e5a0-3417-4373-8d49-a7c34e0fdb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:35.406173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.407315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.407765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.408195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.420048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:35.428557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:35.429942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:35.430753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:35.433530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 37/69] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:35.439344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.467444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 33/70] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:35.468561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36e5a0-3417-4373-8d49-a7c34e0fdb35 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.544138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40c5c9-ea59-42b7-976b-859d356fa58a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.570777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36e5a0-3417-4373-8d49-a7c34e0fdb35 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.582515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.583784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.584291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.584843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.602635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:35.604596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:35.606741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 38/71] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:35.607852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.609463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.610148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.610597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.611011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.611923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:35.612385 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:35.614196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:35.623794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 34/72] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:35.625151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.721705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23443327-7179-4c9e-b1df-4027ed216d81 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.751123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40bcb4-e389-4730-a888-85e88c9d9fd9 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.762022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.762869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.763152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.763937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.774501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:35.782693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:35.783692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 39/73] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:35.784845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:35.809404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.810521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.811027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.811531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.889676 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddef20-d2e6-49aa-a16e-c28a354f7afa None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:35.908190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:35.909743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:35.926226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04220280-ff50-441b-8983-3ff3fd98485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:35.927664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04220280-ff50-441b-8983-3ff3fd98485d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.928133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04220280-ff50-441b-8983-3ff3fd98485d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.928601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04220280-ff50-441b-8983-3ff3fd98485d None admin] PATH_INFO: `/v3/users/f6614ed1ab4445508051598d04c940bf` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.930545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04220280-ff50-441b-8983-3ff3fd98485d None admin] RBAC: Authorizing `identity:get_user(user_id=f6614ed1ab4445508051598d04c940bf)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:35.933226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04220280-ff50-441b-8983-3ff3fd98485d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:35.935128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 40/74] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/users/f6614ed1ab4445508051598d04c940bf => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:35.936757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6d469-b31e-466a-b79f-0980576f1051 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.937192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6d469-b31e-466a-b79f-0980576f1051 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.937647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6d469-b31e-466a-b79f-0980576f1051 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.939361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 41/75] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:35.944018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f38331-9e4a-4aa5-9d89-7d6d7a05ee7f None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:35.944701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f38331-9e4a-4aa5-9d89-7d6d7a05ee7f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:35.945270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f38331-9e4a-4aa5-9d89-7d6d7a05ee7f None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:35.947132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 42/76] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:35.948019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f38331-9e4a-4aa5-9d89-7d6d7a05ee7f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.024313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 35/77] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:36.025777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.067165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f38331-9e4a-4aa5-9d89-7d6d7a05ee7f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.101645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd08350-bed0-4fd3-96e3-a9ebb00d3562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.102353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08350-bed0-4fd3-96e3-a9ebb00d3562 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.102642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08350-bed0-4fd3-96e3-a9ebb00d3562 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.102918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08350-bed0-4fd3-96e3-a9ebb00d3562 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.103704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd08350-bed0-4fd3-96e3-a9ebb00d3562 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:36.104022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd08350-bed0-4fd3-96e3-a9ebb00d3562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:36.105524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd08350-bed0-4fd3-96e3-a9ebb00d3562 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:36.114135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 43/78] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:37:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:36.115513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee5a6-1fef-4063-8bf1-d7ccad194264 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.115806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee5a6-1fef-4063-8bf1-d7ccad194264 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.116088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ee5a6-1fef-4063-8bf1-d7ccad194264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.134279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-387ee5a6-1fef-4063-8bf1-d7ccad194264 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:36.135705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-387ee5a6-1fef-4063-8bf1-d7ccad194264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:36.136299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b861fd-b853-44c7-a8da-f048efae3293 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.184185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.185132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.185576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.186022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.248822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 44/79] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:36.248822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8b77-8ef6-418d-90c8-0b40e8f6bedc None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.248822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8b77-8ef6-418d-90c8-0b40e8f6bedc None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.248822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8b77-8ef6-418d-90c8-0b40e8f6bedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.273550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-857f8b77-8ef6-418d-90c8-0b40e8f6bedc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:36.274776 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-857f8b77-8ef6-418d-90c8-0b40e8f6bedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:36.275890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:36.277621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:36.361277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 45/80] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:36.362696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f8b77-8ef6-418d-90c8-0b40e8f6bedc None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.419780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 36/81] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:36.421252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.514912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f8b77-8ef6-418d-90c8-0b40e8f6bedc None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.567381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.568766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.569186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.569651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.571704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:36.573380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e142086-9d68-4d98-8940-c4e0a5f85c9f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.574378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:36.577337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 46/82] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:36.578405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.626241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.627623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.628085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.628548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.630563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:36.632686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:36.635691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 37/83] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:36.637129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.708511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160f16-10be-4bb2-b92c-5d92ba9b66ca None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.743202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.744018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.744301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.744580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.756895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:36.757131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58ca10-ac1b-4077-935c-b886a7e7b42a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:36.764105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:36.765478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:36.766287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 47/84] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:36.768050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.814903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.815809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.816173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.816467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.817806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:36.819288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:36.821040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 38/85] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:36.821842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.874502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ba4b0-849e-4cb5-90b8-196fe6aec522 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.910925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.911855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.912140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.912418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.913823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:36.915361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:36.917357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 48/86] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:36.918250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:36.936687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a8409-2a5b-45c2-815c-7425b71fbd3f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:36.975419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:36.976099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:36.976381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:36.976662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:36.977580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=9a3eded2657c435894f06d8e9f184826) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:36.977896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:36.979534 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:36.985638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 39/87] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/role_assignments?user.id=9a3eded2657c435894f06d8e9f184826&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=4e8a255633994167b69cee0662997d5e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:36.986533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:37.063679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cf3be-f54a-4e11-b6f8-fc1fc4b61dd5 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.110980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.112040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.112372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.112758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.149080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf19d1-8321-4cf7-9c29-7be621e5b1df None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.205863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.206970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.207461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.207904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.208850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:37.209381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:37.210523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:37.212145 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:37.212288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:37.232925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 40/88] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:37.232925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:37.339178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 49/89] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:37.340697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:37.358921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f5b5-d6f5-4f1e-b510-c2d230ebb9df None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.393373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.394056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.394339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.394637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.395349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:37.396861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:37.427492 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 41/90] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:37:37 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:37.434569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:37.520540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69055e-d1ee-47b2-a24b-2b8286b9800b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.561538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.562019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.562434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.562816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.563801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=f6614ed1ab4445508051598d04c940bf) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:37.564214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:37.565861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:37.572775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 50/91] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:37 2022] GET /identity/v3/role_assignments?user.id=f6614ed1ab4445508051598d04c940bf&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:37.574021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:37.602787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633c96b-1f0b-47bf-bbeb-1bff72ea161d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.651937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.653856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.654657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.655306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.657992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:37.662754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:37.666042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 42/92] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:37 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:37.674764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:37.701314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3115e8d-f916-44bb-8646-9100e9320bed None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.748003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.748952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.749238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.749520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.752006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:37.752509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:37.755019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 51/93] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:37 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:37.760929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:37.866189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fdce-d68c-47df-a0e4-92c2cf36b0fc None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.880432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4696c2-b557-4c11-997e-d7c053825ec3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:37.924037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.924981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.925355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.925709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.927086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] RBAC: Adding query filter params (role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=9a3eded2657c435894f06d8e9f184826) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:37.927086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:37.928953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:37.935376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:37.936077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:37.936372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:37.936649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:37.937736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 52/94] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:37 2022] GET /identity/v3/role_assignments?user.id=9a3eded2657c435894f06d8e9f184826&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:38.074970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:38.078201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:38.226191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 43/95] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:38.232243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:38.383349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f56aa-7b9b-45db-8183-b59bd7105c5e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:38.441977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810e19-7f41-4c19-b3cd-14cda163b94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:38.443458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810e19-7f41-4c19-b3cd-14cda163b94b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:38.443936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810e19-7f41-4c19-b3cd-14cda163b94b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:38.444431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810e19-7f41-4c19-b3cd-14cda163b94b None admin] PATH_INFO: `/v3/projects/1d997cedac8b4e418da7c7d43e99a347` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:38.446782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92810e19-7f41-4c19-b3cd-14cda163b94b None admin] RBAC: Authorizing `identity:get_project(project_id=1d997cedac8b4e418da7c7d43e99a347)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:38.449740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92810e19-7f41-4c19-b3cd-14cda163b94b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:38.452941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 53/96] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:38 2022] GET /identity/v3/projects/1d997cedac8b4e418da7c7d43e99a347 => generated 312 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:38.459762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:38.590362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ded9d-ceeb-41d2-8ce1-91a7bc020b2e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:38.640604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ce282-ee08-456d-beb9-860ebcf6c351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:38.641333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce282-ee08-456d-beb9-860ebcf6c351 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:38.641621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce282-ee08-456d-beb9-860ebcf6c351 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:38.641896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce282-ee08-456d-beb9-860ebcf6c351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:38.642859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ce282-ee08-456d-beb9-860ebcf6c351 None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=1d997cedac8b4e418da7c7d43e99a347, user.id=9a3eded2657c435894f06d8e9f184826) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:38.643176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ce282-ee08-456d-beb9-860ebcf6c351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:38.644722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ce282-ee08-456d-beb9-860ebcf6c351 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:38.655466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 44/97] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:38 2022] GET /identity/v3/role_assignments?user.id=9a3eded2657c435894f06d8e9f184826&scope.project.id=1d997cedac8b4e418da7c7d43e99a347&role.id=4e8a255633994167b69cee0662997d5e => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:40.706034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ed90b-99fc-4ae0-9e95-e8494e223fcb None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:40.706521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ed90b-99fc-4ae0-9e95-e8494e223fcb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:40.706903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ed90b-99fc-4ae0-9e95-e8494e223fcb None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:40.709457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 54/98] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:40.716320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9eb87-b3a0-498b-85c9-3650ac920ede None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:40.716881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9eb87-b3a0-498b-85c9-3650ac920ede None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:40.717294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9eb87-b3a0-498b-85c9-3650ac920ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:40.754552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-52d9eb87-b3a0-498b-85c9-3650ac920ede None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:40.756558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-52d9eb87-b3a0-498b-85c9-3650ac920ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:40.902293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 45/99] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:40.942472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ed90b-99fc-4ae0-9e95-e8494e223fcb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:40.981066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ffc9-f7a0-4f3e-b89d-5f21d5aea17f None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:40.981321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ffc9-f7a0-4f3e-b89d-5f21d5aea17f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:40.981625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ffc9-f7a0-4f3e-b89d-5f21d5aea17f None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:40.982927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 46/100] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:40.990562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079b2af-d926-455b-a923-10c48e052cf6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:40.990945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079b2af-d926-455b-a923-10c48e052cf6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:40.991176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079b2af-d926-455b-a923-10c48e052cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.016739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5079b2af-d926-455b-a923-10c48e052cf6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:41.022737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5079b2af-d926-455b-a923-10c48e052cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:41.177532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ed90b-99fc-4ae0-9e95-e8494e223fcb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:41.234291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 47/101] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 04 14:37:41.239888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:41.241255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.241778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.242278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.263911 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:41.277416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:41.278394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:41.279179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:41.282005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 55/102] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:41.288569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079b2af-d926-455b-a923-10c48e052cf6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:41.305262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:41.413065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fc879-c468-467f-b3be-865c989780ac None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:41.457483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:41.458738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.459220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.459690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.482968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:41.497815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079b2af-d926-455b-a923-10c48e052cf6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:41.499480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:41.501311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:41.502791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 56/103] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:41.528766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:41.578226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:41.579786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.580497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.581464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.583390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:41.584551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:41.587540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:41.608099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 48/104] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:41.615129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:41.777440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c439b5-3dba-4b5c-b7e0-eb5a41066189 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:41.812392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef235205-2b5f-4901-911c-da34b0b5887b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:41.813104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef235205-2b5f-4901-911c-da34b0b5887b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.813544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef235205-2b5f-4901-911c-da34b0b5887b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.813982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef235205-2b5f-4901-911c-da34b0b5887b None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.814901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef235205-2b5f-4901-911c-da34b0b5887b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:41.815402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef235205-2b5f-4901-911c-da34b0b5887b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:41.817080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef235205-2b5f-4901-911c-da34b0b5887b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:41.826567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 57/105] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:41.828210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b7e7-f2a4-43f4-afa2-47748acbe280 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.828701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b7e7-f2a4-43f4-afa2-47748acbe280 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.829263 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0b7e7-f2a4-43f4-afa2-47748acbe280 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.830782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 58/106] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:41.832106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3dee-5eb7-47f9-9118-4d1d0acb246e None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.832559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3dee-5eb7-47f9-9118-4d1d0acb246e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.833032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b3dee-5eb7-47f9-9118-4d1d0acb246e None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.834451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 59/107] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:41.838514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40da07c-9ce9-41f6-83b9-6e312c398189 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:41.840465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3aba6f-0fd9-4d75-a13d-c2cd5b971676 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.840960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3aba6f-0fd9-4d75-a13d-c2cd5b971676 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.841428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3aba6f-0fd9-4d75-a13d-c2cd5b971676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.859708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0c3aba6f-0fd9-4d75-a13d-c2cd5b971676 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:41.861297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0c3aba6f-0fd9-4d75-a13d-c2cd5b971676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:41.898011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:41.899421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.900142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.900812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.903195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:41.906025 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:41.909342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 49/108] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:41 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:41.911064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:41.937198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 60/109] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:41.938670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07efe2-8cd6-47cb-a8d7-f9e4f21b4e80 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:41.944823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07efe2-8cd6-47cb-a8d7-f9e4f21b4e80 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:41.944823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07efe2-8cd6-47cb-a8d7-f9e4f21b4e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:41.960413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1b07efe2-8cd6-47cb-a8d7-f9e4f21b4e80 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:41.961753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1b07efe2-8cd6-47cb-a8d7-f9e4f21b4e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:42.036081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 61/110] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:42.036879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07efe2-8cd6-47cb-a8d7-f9e4f21b4e80 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.070476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9e9f1-597e-43f0-be0b-a00cd6dac1d7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.114920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da459365-0f17-4528-a922-8a723182444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.116407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da459365-0f17-4528-a922-8a723182444f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.116933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da459365-0f17-4528-a922-8a723182444f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.117396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da459365-0f17-4528-a922-8a723182444f None admin] PATH_INFO: `/v3/users/f6614ed1ab4445508051598d04c940bf` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.119445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da459365-0f17-4528-a922-8a723182444f None admin] RBAC: Authorizing `identity:get_user(user_id=f6614ed1ab4445508051598d04c940bf)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:42.122102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da459365-0f17-4528-a922-8a723182444f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:42.125001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 50/111] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:41 2022] GET /identity/v3/users/f6614ed1ab4445508051598d04c940bf => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:42.130183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eda78-340b-45d0-9f70-01a6c9fce7c1 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.130475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eda78-340b-45d0-9f70-01a6c9fce7c1 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.133312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eda78-340b-45d0-9f70-01a6c9fce7c1 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.136510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 51/112] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:42.139078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eda78-340b-45d0-9f70-01a6c9fce7c1 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.167476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07efe2-8cd6-47cb-a8d7-f9e4f21b4e80 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.217369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.218685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.219140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.219592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.221613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:42.223990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:42.226876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 62/113] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:42 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:42.228184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.296360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eda78-340b-45d0-9f70-01a6c9fce7c1 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.347519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.348924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.349561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.350048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.351538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:42.353386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:42.355747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 52/114] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:42.356896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.374354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763bb9-6474-46d4-911c-ecf64ec605ce None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.426485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52f226-9d3b-4643-bf0c-6059a1b0b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.427635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f226-9d3b-4643-bf0c-6059a1b0b2fa None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.428083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f226-9d3b-4643-bf0c-6059a1b0b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.428538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52f226-9d3b-4643-bf0c-6059a1b0b2fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.444565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52f226-9d3b-4643-bf0c-6059a1b0b2fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-5c52f226-9d3b-4643-bf0c-6059a1b0b2fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:42.458686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:42.459704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:37:42.460466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:42.461156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 63/115] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:42.462752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f99e-3385-43c1-aecc-2e9f44d41a4d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.463239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f99e-3385-43c1-aecc-2e9f44d41a4d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.463843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f99e-3385-43c1-aecc-2e9f44d41a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.481783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a603f99e-3385-43c1-aecc-2e9f44d41a4d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:42.483001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a603f99e-3385-43c1-aecc-2e9f44d41a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:42.491270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b0197-3413-45dc-900c-e3dcac77cdf0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.526645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.527523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.527799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.528105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.529334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:42.530823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:42.532584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 53/116] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:42 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:42.533353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.555040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 64/117] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:42.556329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603f99e-3385-43c1-aecc-2e9f44d41a4d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.655316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590200e4-9156-42cc-9411-ffc259e9a1ef None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.697866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603f99e-3385-43c1-aecc-2e9f44d41a4d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.704699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f04d92-ca40-4269-84f6-6029ca3763cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.706173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f04d92-ca40-4269-84f6-6029ca3763cb None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.706629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f04d92-ca40-4269-84f6-6029ca3763cb None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.707077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f04d92-ca40-4269-84f6-6029ca3763cb None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46/users/9a3eded2657c435894f06d8e9f184826/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.710155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f04d92-ca40-4269-84f6-6029ca3763cb None admin] RBAC: Authorizing `identity:create_grant(project_id=c7555237bf4842dd8f28617418adec46, user_id=9a3eded2657c435894f06d8e9f184826, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:42.712542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f04d92-ca40-4269-84f6-6029ca3763cb None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:42.731023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 54/118] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:37:42 2022] PUT /identity/v3/projects/c7555237bf4842dd8f28617418adec46/users/9a3eded2657c435894f06d8e9f184826/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:42.733470 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888af49-860e-441a-b14a-fda0cc6f36a9 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.733954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888af49-860e-441a-b14a-fda0cc6f36a9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.734406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888af49-860e-441a-b14a-fda0cc6f36a9 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.736684 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 55/119] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:42.737992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888af49-860e-441a-b14a-fda0cc6f36a9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.757474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.758180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.758472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.758752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.759529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:42.761073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:42.790677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 65/120] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:37:42 2022] POST /identity/v3/projects => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:42.792113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.898724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888af49-860e-441a-b14a-fda0cc6f36a9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:42.939004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:42.940229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:42.940727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:42.941244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:42.942284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:42.942950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:42.944766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:42.956177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 56/121] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:42.957513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:42.968878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeb8d7-662f-4c0b-b701-a8568db17422 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:43.038897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9e7a8-515c-4a89-acfa-0cf0ae2739f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:43.040028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e7a8-515c-4a89-acfa-0cf0ae2739f8 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:43.040898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e7a8-515c-4a89-acfa-0cf0ae2739f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:43.041303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9e7a8-515c-4a89-acfa-0cf0ae2739f8 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46/users/f6614ed1ab4445508051598d04c940bf/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:43.046163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9e7a8-515c-4a89-acfa-0cf0ae2739f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7555237bf4842dd8f28617418adec46, user_id=f6614ed1ab4445508051598d04c940bf, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:43.048837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9e7a8-515c-4a89-acfa-0cf0ae2739f8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:43.076608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 66/122] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:37:42 2022] PUT /identity/v3/projects/c7555237bf4842dd8f28617418adec46/users/f6614ed1ab4445508051598d04c940bf/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:43.079110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b518e-e211-40f4-9dc1-9117a173fcd9 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:43.079427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b518e-e211-40f4-9dc1-9117a173fcd9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:43.079710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b518e-e211-40f4-9dc1-9117a173fcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:43.098322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6d3b518e-e211-40f4-9dc1-9117a173fcd9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:43.099673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6d3b518e-e211-40f4-9dc1-9117a173fcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:43.130061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824b84d-e069-47ff-9312-83d073ad02e6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:43.192332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:43.193663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:43.193885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 67/123] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:43.194050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:43.194628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:43.195565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b518e-e211-40f4-9dc1-9117a173fcd9 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:43.231246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:43.260745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:43.261775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:43.262555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:43.263158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 57/124] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:37:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:43.264909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:43.475922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b518e-e211-40f4-9dc1-9117a173fcd9 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:43.477456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2f4c5-2192-4121-a9aa-9d0dc3cc02e2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:43.526244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:43.527859 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:43.528990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:43.529492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] PATH_INFO: `/v3/users/294d56c712ed49e3b8acbd7739911193` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:43.552260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=294d56c712ed49e3b8acbd7739911193)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:43.555860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:43.559487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 68/125] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:43 2022] GET /identity/v3/users/294d56c712ed49e3b8acbd7739911193 => generated 289 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:43.561492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:43.563268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:43.569739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:43.569739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:43.569739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:43.610461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:43.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:43.642733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:43.644303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 58/126] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:43.649460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:43.750917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c027a-4482-426f-a48e-e92dc5b18ae9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:43.814181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:43.814972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:43.815168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:43.815451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:43.816231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:43.816542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:43.818017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:43.835693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cfba9-d050-485b-8c70-05e699907b5e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:43.848517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 69/127] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:37:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:43.854265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:43.874131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:43.874808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:43.875096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:43.875375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:43.957275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:43.958812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:44.047077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edc5b0-f189-403a-845f-9bd1e9b342d7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:44.127078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:44.128228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:44.128708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:44.129223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:44.130612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] RBAC: Adding query filter params (name=member) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:44.131172 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:44.133740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:44.158744 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 70/128] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:44.160381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:44.187610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 59/129] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:44.189086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:44.355498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac565-2ec2-4765-8e55-1ebf96f73937 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:44.362240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6942c1-99a0-417b-868d-911f3676b48a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:44.430899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69453963-71a8-480f-9784-58b565863c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:44.432434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69453963-71a8-480f-9784-58b565863c3c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:44.433952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69453963-71a8-480f-9784-58b565863c3c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:44.434436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69453963-71a8-480f-9784-58b565863c3c None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:44.437272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69453963-71a8-480f-9784-58b565863c3c None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:44.439986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69453963-71a8-480f-9784-58b565863c3c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:44.444103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:44.445671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:44.446149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:44.446659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:44.450089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:44.452753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:44.453659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 60/130] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:44 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:44.454573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69453963-71a8-480f-9784-58b565863c3c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:44.455774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 71/131] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:44 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:44.465534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:44.610502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1095eb-b25a-4b87-be0f-3f733edeb354 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:44.655779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:44.656752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:44.657149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:44.657433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] PATH_INFO: `/v3/projects/1d997cedac8b4e418da7c7d43e99a347` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:44.661695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] RBAC: Authorizing `identity:get_project(project_id=1d997cedac8b4e418da7c7d43e99a347)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:44.663514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:44.665696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 72/132] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:44 2022] GET /identity/v3/projects/1d997cedac8b4e418da7c7d43e99a347 => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:44.666934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:44.710076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69453963-71a8-480f-9784-58b565863c3c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:44.771962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:44.774020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:44.774408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:44.774909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:44.777207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:44.780293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:44.783691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 61/133] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:44 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:44.790182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:44.850085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983287ad-f9c3-4f90-9190-a8aefc412730 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:44.892340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:44.893651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:44.894143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:44.894609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:44.896215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:44.898349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:44.900493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 73/134] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:44 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:44.901761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:45.017165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aaf8c-f57c-438b-9015-f7f8b637c176 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:45.055360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:45.056418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:45.056931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:45.057475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:45.058557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] RBAC: Adding query filter params (role.id=070af100905e4b06b408f229d26a3bdb, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=294d56c712ed49e3b8acbd7739911193) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:45.059106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:45.060823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:45.063974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68f973-2307-42c5-89fd-e77befc08a32 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:45.067960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 74/135] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:44 2022] GET /identity/v3/role_assignments?user.id=294d56c712ed49e3b8acbd7739911193&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=070af100905e4b06b408f229d26a3bdb => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:45.069148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:45.138227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8463c5b-2b44-408c-b7ca-49c24b796db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:45.139163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8463c5b-2b44-408c-b7ca-49c24b796db5 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:45.139447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8463c5b-2b44-408c-b7ca-49c24b796db5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:45.139724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8463c5b-2b44-408c-b7ca-49c24b796db5 None admin] PATH_INFO: `/v3/projects/1d997cedac8b4e418da7c7d43e99a347/users/9a3eded2657c435894f06d8e9f184826/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:45.143716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8463c5b-2b44-408c-b7ca-49c24b796db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d997cedac8b4e418da7c7d43e99a347, user_id=9a3eded2657c435894f06d8e9f184826, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:45.145250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8463c5b-2b44-408c-b7ca-49c24b796db5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:45.175499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 62/136] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:37:44 2022] PUT /identity/v3/projects/1d997cedac8b4e418da7c7d43e99a347/users/9a3eded2657c435894f06d8e9f184826/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:45.185109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba8115-51d7-4b7d-91ad-aae113655229 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:45.225754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7dc87-c2ee-447d-bc23-db5282f21c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:45.227130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dc87-c2ee-447d-bc23-db5282f21c8c None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:45.227617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dc87-c2ee-447d-bc23-db5282f21c8c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:45.228099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7dc87-c2ee-447d-bc23-db5282f21c8c None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46/users/9a3eded2657c435894f06d8e9f184826/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:45.231056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7dc87-c2ee-447d-bc23-db5282f21c8c None admin] RBAC: Authorizing `identity:create_grant(project_id=c7555237bf4842dd8f28617418adec46, user_id=9a3eded2657c435894f06d8e9f184826, role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:45.233009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7dc87-c2ee-447d-bc23-db5282f21c8c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:45.248006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 75/137] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:37:45 2022] PUT /identity/v3/projects/c7555237bf4842dd8f28617418adec46/users/9a3eded2657c435894f06d8e9f184826/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:45.890923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972c7ae-6d7e-41a4-aa23-2fa257850d2b None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:45.891414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972c7ae-6d7e-41a4-aa23-2fa257850d2b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:45.891557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972c7ae-6d7e-41a4-aa23-2fa257850d2b None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:45.892960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 63/138] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:45.907523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde2b9c-782c-4677-ae74-9bf99b6b25ae None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:45.908550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde2b9c-782c-4677-ae74-9bf99b6b25ae None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:45.909107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde2b9c-782c-4677-ae74-9bf99b6b25ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:45.945149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0dde2b9c-782c-4677-ae74-9bf99b6b25ae None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:45.947470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0dde2b9c-782c-4677-ae74-9bf99b6b25ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:46.044152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 76/139] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:46.085784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972c7ae-6d7e-41a4-aa23-2fa257850d2b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:46.225593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972c7ae-6d7e-41a4-aa23-2fa257850d2b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:46.270638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd1630-585f-43da-8849-c8525a8f0bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:46.271448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd1630-585f-43da-8849-c8525a8f0bf1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:46.271731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd1630-585f-43da-8849-c8525a8f0bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:46.272005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd1630-585f-43da-8849-c8525a8f0bf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:46.277640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd1630-585f-43da-8849-c8525a8f0bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:46.280117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd1630-585f-43da-8849-c8525a8f0bf1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:46.282719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 64/140] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:46.287991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde2b9c-782c-4677-ae74-9bf99b6b25ae None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:46.304277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffa3ce-0490-4e29-9cfa-f4935e863c5e None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:46.304578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffa3ce-0490-4e29-9cfa-f4935e863c5e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:46.304863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffa3ce-0490-4e29-9cfa-f4935e863c5e None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:46.308869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 65/141] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:46.312193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1bfef-d4ac-44a5-829b-5624505350f4 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:46.312443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1bfef-d4ac-44a5-829b-5624505350f4 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:46.312653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1bfef-d4ac-44a5-829b-5624505350f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:46.339734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-26f1bfef-d4ac-44a5-829b-5624505350f4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:46.345220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-26f1bfef-d4ac-44a5-829b-5624505350f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:46.486048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde2b9c-782c-4677-ae74-9bf99b6b25ae None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:46.514999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 66/142] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:46.561756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:46.562480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:46.562791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:46.563063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:46.563796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:46.565338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:46.571333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1bfef-d4ac-44a5-829b-5624505350f4 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:46.592499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 77/143] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:37:46 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:46.719363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1bfef-d4ac-44a5-829b-5624505350f4 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:46.757053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:46.757872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:46.758158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:46.758433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:46.770579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:46.778825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:46.779768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:46.780551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 67/144] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:46.788202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:46.975400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b40e6d-ffb6-426f-876d-54a13340c497 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:47.011247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:47.011854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:47.012139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:47.012415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:47.013272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] RBAC: Adding query filter params (name=member) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:47.013577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:47.015159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:47.024810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 78/145] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:47.033942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:47.177068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81ed86-8e81-4e64-aea1-8213ee9438c6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:47.225469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a649a-0469-4737-9c87-9e573ed9ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:47.226379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a649a-0469-4737-9c87-9e573ed9ca73 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:47.226664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a649a-0469-4737-9c87-9e573ed9ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:47.226940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a649a-0469-4737-9c87-9e573ed9ca73 None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:47.228182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a649a-0469-4737-9c87-9e573ed9ca73 None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:47.229723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a649a-0469-4737-9c87-9e573ed9ca73 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:47.231411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 68/146] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:47 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:47.237513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:47.396079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e7839-bdce-432e-b58e-59dd91da6d80 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:47.434436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566f8e-2409-478e-9674-efd2dfb29a82 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:47.434753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566f8e-2409-478e-9674-efd2dfb29a82 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:47.434993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:47.435104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566f8e-2409-478e-9674-efd2dfb29a82 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:47.435665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:47.435951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:47.436225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:47.441003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 69/147] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:47.454755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3da03-b322-44fd-849d-af83c4159a74 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:47.455067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3da03-b322-44fd-849d-af83c4159a74 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:47.455306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3da03-b322-44fd-849d-af83c4159a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:47.485165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-24f3da03-b322-44fd-849d-af83c4159a74 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:47.486372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-24f3da03-b322-44fd-849d-af83c4159a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:47.514601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:47.516125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:47.621447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 79/148] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:37:47.636392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 70/149] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:37:47.638671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:47.679009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3da03-b322-44fd-849d-af83c4159a74 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:47.821585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ddb67f-e342-49c2-bc70-824b5389c295 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:47.869608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:47.870932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:47.871440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:47.872011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:47.873565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:47.875268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:47.877347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 80/150] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:47 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:47.884954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:47.891165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3da03-b322-44fd-849d-af83c4159a74 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:47.966853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:47.967688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:47.967975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:47.968251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:47.989561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:47.999220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:48.000175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:48.000974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 71/151] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:48.009854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:48.041713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f5760-0d65-4ab6-bf42-4c04de729234 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:48.091003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:48.092072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:48.092547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:48.093039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:48.094104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=9a3eded2657c435894f06d8e9f184826) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:48.094617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:48.096283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:48.103053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 81/152] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:47 2022] GET /identity/v3/role_assignments?user.id=9a3eded2657c435894f06d8e9f184826&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=4e8a255633994167b69cee0662997d5e => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:48.159902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb66a4-7e1e-4a37-ab5c-522626e2925b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:48.213917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:48.214688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:48.214950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:48.215200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:48.216003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:48.216331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:48.217855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:48.230226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 72/153] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:48.236221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:48.399301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0312c-28e7-414d-9316-be6fb07de595 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:48.447588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6979bbd-7c7f-4632-a65c-ef34290bdc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:48.448938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6979bbd-7c7f-4632-a65c-ef34290bdc76 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:48.449410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6979bbd-7c7f-4632-a65c-ef34290bdc76 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:48.449871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6979bbd-7c7f-4632-a65c-ef34290bdc76 None admin] PATH_INFO: `/v3/users/f6614ed1ab4445508051598d04c940bf` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:48.451378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6979bbd-7c7f-4632-a65c-ef34290bdc76 None admin] RBAC: Authorizing `identity:get_user(user_id=f6614ed1ab4445508051598d04c940bf)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:48.453132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6979bbd-7c7f-4632-a65c-ef34290bdc76 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:48.455468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 82/154] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:48 2022] GET /identity/v3/users/f6614ed1ab4445508051598d04c940bf => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:48.461375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:48.606901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef12b3-b17c-4e15-bd01-e20f400ccaae None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:48.657317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:48.657989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:48.658274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:48.658552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:48.687323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077576b1-7122-4f41-a39a-206b324e4fc7 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:48.687937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077576b1-7122-4f41-a39a-206b324e4fc7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:48.688422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077576b1-7122-4f41-a39a-206b324e4fc7 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:48.690104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 83/155] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:48.703682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945765-907d-44d8-b327-76dca7271643 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:48.704273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945765-907d-44d8-b327-76dca7271643 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:48.704760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945765-907d-44d8-b327-76dca7271643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:48.737889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-bd945765-907d-44d8-b327-76dca7271643 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:48.739544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bd945765-907d-44d8-b327-76dca7271643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:48.756841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:48.757184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:48.848615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 84/156] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:37:48.876898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 73/157] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:48.882294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945765-907d-44d8-b327-76dca7271643 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:48.887164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:49.021091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945765-907d-44d8-b327-76dca7271643 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:49.069073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:49.070789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.071263 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.071746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.073162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b8523-44aa-4619-9605-b09c57a738ce None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:49.074151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:49.075823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:49.077828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 85/158] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:48 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:49.083797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:49.143484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:49.144573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.145109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.145740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.168735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:49.187493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:49.188433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:37:49.189230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:49.193987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 74/159] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:49.200395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:49.204196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8f482-1469-43fd-bfa4-3fd1ddfae2fd None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:49.245351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:49.247217 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.247853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.249537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.250593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=f6614ed1ab4445508051598d04c940bf) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:49.251344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:49.253846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:49.271982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 86/160] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:49 2022] GET /identity/v3/role_assignments?user.id=f6614ed1ab4445508051598d04c940bf&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:49.383331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bba1bd-78f8-4326-9529-f9dd1b200a6a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:49.429490 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6d2f-094e-468b-abb1-cb06a71194fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:49.430760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6d2f-094e-468b-abb1-cb06a71194fb None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.431337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6d2f-094e-468b-abb1-cb06a71194fb None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.431819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6d2f-094e-468b-abb1-cb06a71194fb None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.432943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6d2f-094e-468b-abb1-cb06a71194fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:49.433527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6d2f-094e-468b-abb1-cb06a71194fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:49.435269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6d2f-094e-468b-abb1-cb06a71194fb None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:49.449140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 75/161] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:49.456562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:49.550633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f128a-c642-4024-bada-ad2e26a7e1b0 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.551417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f128a-c642-4024-bada-ad2e26a7e1b0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.552178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f128a-c642-4024-bada-ad2e26a7e1b0 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.554114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 76/162] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:49.559960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd92cc-da69-439e-aee9-1b1d9e862228 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.560529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd92cc-da69-439e-aee9-1b1d9e862228 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.561015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd92cc-da69-439e-aee9-1b1d9e862228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.588751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b9cd92cc-da69-439e-aee9-1b1d9e862228 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:49.591309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b9cd92cc-da69-439e-aee9-1b1d9e862228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:49.599276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df06a-b4cc-4bc6-96ce-bb35c0f49db9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:49.672427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:49.673383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.673657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.673922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] PATH_INFO: `/v3/users/294d56c712ed49e3b8acbd7739911193` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.675234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] RBAC: Authorizing `identity:get_user(user_id=294d56c712ed49e3b8acbd7739911193)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:49.676732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:49.680918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 87/163] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:49 2022] GET /identity/v3/users/294d56c712ed49e3b8acbd7739911193 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:49.687092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:49.757077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 77/164] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:37:49.813464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd92cc-da69-439e-aee9-1b1d9e862228 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:49.868726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a58af5-f181-440e-89b9-c3ae677643f9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:49.946487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:49.947371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:49.947680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:49.947962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:49.949255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:49.950748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:49.956921 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 88/165] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:49 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:49.958861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:49.983450 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd92cc-da69-439e-aee9-1b1d9e862228 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:50.050650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:50.051704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:50.051825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:50.052010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:50.077846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:50.089756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:50.090728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:50.091520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 78/166] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:50.100445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:50.174287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4002db-48e8-40a4-9a64-d4968940d424 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:50.237746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33546151-03a6-4b24-bd3c-079b176905e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:50.238952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33546151-03a6-4b24-bd3c-079b176905e4 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:50.239681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33546151-03a6-4b24-bd3c-079b176905e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:50.239981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33546151-03a6-4b24-bd3c-079b176905e4 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46/users/294d56c712ed49e3b8acbd7739911193/roles/070af100905e4b06b408f229d26a3bdb` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:50.242347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33546151-03a6-4b24-bd3c-079b176905e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7555237bf4842dd8f28617418adec46, user_id=294d56c712ed49e3b8acbd7739911193, role_id=070af100905e4b06b408f229d26a3bdb)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:50.243838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33546151-03a6-4b24-bd3c-079b176905e4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:50.275495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 89/167] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:37:49 2022] PUT /identity/v3/projects/c7555237bf4842dd8f28617418adec46/users/294d56c712ed49e3b8acbd7739911193/roles/070af100905e4b06b408f229d26a3bdb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:50.329837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cac56-bda9-4caf-b4ef-242788f366e8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:50.390445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:50.392579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:50.393184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:50.393671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:50.394712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:50.395240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:50.397091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:50.404010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e0ef-cabd-43a8-b853-d9ea082be790 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:50.404385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e0ef-cabd-43a8-b853-d9ea082be790 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:50.404547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e0ef-cabd-43a8-b853-d9ea082be790 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:50.406108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 90/168] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:50.416290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 79/169] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:50.422506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74415096-0ab1-48b8-bd50-5c397f5111d0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:50.422795 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74415096-0ab1-48b8-bd50-5c397f5111d0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:50.423068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74415096-0ab1-48b8-bd50-5c397f5111d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:50.424193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:50.462192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-74415096-0ab1-48b8-bd50-5c397f5111d0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:50.464040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-74415096-0ab1-48b8-bd50-5c397f5111d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:50.617195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 91/170] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:50.629288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7807-8036-4afc-b3f1-0446271b5a28 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:50.656384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74415096-0ab1-48b8-bd50-5c397f5111d0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:50.699896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:50.701501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:50.702033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:50.702506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:50.704163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:50.706733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:50.709168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 80/171] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:50 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:50.718218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:50.853533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802da2f-adf7-4289-84a8-7dd679956a09 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:50.890818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:50.891553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:50.891835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:50.892107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:50.900602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74415096-0ab1-48b8-bd50-5c397f5111d0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:51.003153 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec899ec7-7cbb-46fe-8846-d444ac39e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:51.003999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec899ec7-7cbb-46fe-8846-d444ac39e1df None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.004280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec899ec7-7cbb-46fe-8846-d444ac39e1df None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.004549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec899ec7-7cbb-46fe-8846-d444ac39e1df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.031672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec899ec7-7cbb-46fe-8846-d444ac39e1df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-ec899ec7-7cbb-46fe-8846-d444ac39e1df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:51.051179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:51.052219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:51.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 92/172] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:37:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:51.054408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896c0f-1420-4818-b685-01ead5ef1503 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.060238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896c0f-1420-4818-b685-01ead5ef1503 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.060238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896c0f-1420-4818-b685-01ead5ef1503 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.060238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 93/173] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:51.062513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896c0f-1420-4818-b685-01ead5ef1503 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:51.101753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:51.103334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:51.181576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896c0f-1420-4818-b685-01ead5ef1503 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:51.224673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847a1b1-2def-4c36-b8fc-feaa0e17c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:51.225399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847a1b1-2def-4c36-b8fc-feaa0e17c5e0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.225676 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847a1b1-2def-4c36-b8fc-feaa0e17c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.225953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847a1b1-2def-4c36-b8fc-feaa0e17c5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.227175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847a1b1-2def-4c36-b8fc-feaa0e17c5e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:51.227639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847a1b1-2def-4c36-b8fc-feaa0e17c5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:51.229156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847a1b1-2def-4c36-b8fc-feaa0e17c5e0 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:51.239970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 94/174] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:37:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:51.241871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae8824-6f4a-4e45-ab8c-b412a8fbd8fa None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.242169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae8824-6f4a-4e45-ab8c-b412a8fbd8fa None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.242602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae8824-6f4a-4e45-ab8c-b412a8fbd8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.261570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e2ae8824-6f4a-4e45-ab8c-b412a8fbd8fa None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:51.263210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e2ae8824-6f4a-4e45-ab8c-b412a8fbd8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:51.341325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 95/175] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:51.342202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae8824-6f4a-4e45-ab8c-b412a8fbd8fa None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:51.440301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 81/176] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:51.441378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:51.519696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae8824-6f4a-4e45-ab8c-b412a8fbd8fa None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:51.589313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:51.590258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.590542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.590830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] PATH_INFO: `/v3/users/9a3eded2657c435894f06d8e9f184826` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.592681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] RBAC: Authorizing `identity:get_user(user_id=9a3eded2657c435894f06d8e9f184826)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:51.594274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:51.596166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 96/177] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:51 2022] GET /identity/v3/users/9a3eded2657c435894f06d8e9f184826 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:51.606986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e0738-ad3e-4120-9252-18126cb1c2d8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:51.609287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:51.690739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:51.691559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.691846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.692182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.714844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:51.723026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:51.724105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 82/178] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:51.729546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:51.764369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92610a-b262-4eb9-a446-d1e87eb3fcf4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:51.821605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:51.822957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.823438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.823927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] PATH_INFO: `/v3/projects/1d997cedac8b4e418da7c7d43e99a347` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.825494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] RBAC: Authorizing `identity:get_project(project_id=1d997cedac8b4e418da7c7d43e99a347)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:51.827224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:51.832149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 97/179] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:51 2022] GET /identity/v3/projects/1d997cedac8b4e418da7c7d43e99a347 => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:51.833360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:51.871251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42765a-37a2-4dbe-937b-26659b8a1703 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:51.908905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab4151-c1e2-4121-86a9-cedb083088db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:51.909624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab4151-c1e2-4121-86a9-cedb083088db None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.909910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab4151-c1e2-4121-86a9-cedb083088db None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.910188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab4151-c1e2-4121-86a9-cedb083088db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.954450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c5034-b9df-4690-b044-09c07acdcba6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:51.989419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab4151-c1e2-4121-86a9-cedb083088db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:51.990849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab4151-c1e2-4121-86a9-cedb083088db None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:51.994505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:51.995201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:51.995481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:51.995754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:51.996537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:51.996886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:51.998347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:52.007669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 98/180] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:52.012529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:52.118649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 83/181] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:52.118649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca955f1-c27d-4f7a-b8cd-536170c87280 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.118649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca955f1-c27d-4f7a-b8cd-536170c87280 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.118649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca955f1-c27d-4f7a-b8cd-536170c87280 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.118649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 84/182] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:52.118649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb32a7-e10d-427d-a483-be500b302e62 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.118649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb32a7-e10d-427d-a483-be500b302e62 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.119258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb32a7-e10d-427d-a483-be500b302e62 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.119688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 85/183] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:52.120250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb32a7-e10d-427d-a483-be500b302e62 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:52.133345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d717f-4925-4c9b-b742-7c78e2d73f9e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:52.176312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:52.177230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.177513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.177787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.178630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=1d997cedac8b4e418da7c7d43e99a347, user.id=9a3eded2657c435894f06d8e9f184826) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:52.178966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:52.181394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:52.192833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 99/184] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/role_assignments?user.id=9a3eded2657c435894f06d8e9f184826&scope.project.id=1d997cedac8b4e418da7c7d43e99a347&role.id=4e8a255633994167b69cee0662997d5e => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:52.192833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:52.252952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb32a7-e10d-427d-a483-be500b302e62 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:52.316158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855b62-3ac3-4821-8906-2b2fb61a9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:52.320232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855b62-3ac3-4821-8906-2b2fb61a9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.320880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855b62-3ac3-4821-8906-2b2fb61a9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.321357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855b62-3ac3-4821-8906-2b2fb61a9aa3 None admin] PATH_INFO: `/v3/users/0bef02839fb44788b776f6630efc17ac` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.341540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855b62-3ac3-4821-8906-2b2fb61a9aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=0bef02839fb44788b776f6630efc17ac)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:52.344492 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855b62-3ac3-4821-8906-2b2fb61a9aa3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:52.347623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 86/185] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/users/0bef02839fb44788b776f6630efc17ac => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:52.349728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052cd9c-7447-4ad5-b714-3918541cbf0a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.350321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052cd9c-7447-4ad5-b714-3918541cbf0a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.350914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052cd9c-7447-4ad5-b714-3918541cbf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.355257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dcc0e-d038-47f4-b3ff-025be9534a2e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:52.374110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b052cd9c-7447-4ad5-b714-3918541cbf0a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:52.376537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b052cd9c-7447-4ad5-b714-3918541cbf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:52.399411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b778fe-1079-458e-b482-9e2f9e843876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:52.400339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b778fe-1079-458e-b482-9e2f9e843876 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.402043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b778fe-1079-458e-b482-9e2f9e843876 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.402325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b778fe-1079-458e-b482-9e2f9e843876 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.403800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b778fe-1079-458e-b482-9e2f9e843876 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:52.406441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b778fe-1079-458e-b482-9e2f9e843876 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:52.407117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 100/186] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:52.411697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c68be-a884-4fdc-93a3-6fdb0b7d7966 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.411697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c68be-a884-4fdc-93a3-6fdb0b7d7966 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.411697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c68be-a884-4fdc-93a3-6fdb0b7d7966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.430340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-fc4c68be-a884-4fdc-93a3-6fdb0b7d7966 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:52.431594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fc4c68be-a884-4fdc-93a3-6fdb0b7d7966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:52.487338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 87/187] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:52.488403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052cd9c-7447-4ad5-b714-3918541cbf0a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:52.538554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 101/188] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:52.539737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c68be-a884-4fdc-93a3-6fdb0b7d7966 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:52.619824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052cd9c-7447-4ad5-b714-3918541cbf0a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:52.661285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c68be-a884-4fdc-93a3-6fdb0b7d7966 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:52.661839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:52.662938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.663329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.664076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.706428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:52.707141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.707421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.707739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.708955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] RBAC: Adding query filter params (role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=9a3eded2657c435894f06d8e9f184826) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:52.709270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:52.711323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:52.725117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 102/189] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/role_assignments?user.id=9a3eded2657c435894f06d8e9f184826&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:52.725117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:52.744714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:52.746283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:52.863059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d642b-350e-403a-907d-7f349c789d89 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:52.893254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 88/190] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:52.899277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:52.907858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:52.909357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:52.909636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:52.909906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:52.911238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:52.913634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:52.916049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 103/191] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:52.916844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.023482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc342e-b2bb-4808-b253-ac0aaf9e84e7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.048263 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730f466-0136-475b-ac2d-a3a2539ae0c4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.069186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.070115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.070391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.070673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.081630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:53.089422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:53.089922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:53.090782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:53.091563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:53.092176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.092176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.092176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.093145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 89/192] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:53.093145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.096973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:53.098506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:53.100454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 104/193] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:52 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:53.101105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.190846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e92c-a42b-4d99-9bc7-17a5bd984f37 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.202580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1636bd-a18a-48bd-84ef-f15dea815030 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.223550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1afc16-3c38-48f7-b001-326faaf16195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.224235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1afc16-3c38-48f7-b001-326faaf16195 None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.224538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1afc16-3c38-48f7-b001-326faaf16195 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.224821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1afc16-3c38-48f7-b001-326faaf16195 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.225566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1afc16-3c38-48f7-b001-326faaf16195 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:53.227038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1afc16-3c38-48f7-b001-326faaf16195 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:53.234609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.235297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.235646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.235875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.236690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:53.237098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:53.238583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:53.247235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 105/194] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:53.256925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 90/195] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:37:53 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:37:53.256925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12c7a6-4a22-4218-b053-6d2474223ba3 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.256925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12c7a6-4a22-4218-b053-6d2474223ba3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.256925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12c7a6-4a22-4218-b053-6d2474223ba3 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.256925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 91/196] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:53.257422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.257509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12c7a6-4a22-4218-b053-6d2474223ba3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.368387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80fe1a-08e6-4dde-a578-fc40310634a3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.369809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12c7a6-4a22-4218-b053-6d2474223ba3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.404037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.404715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.405018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.405179 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b49985-896d-40d4-b47e-c88f88fd3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.405283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.406043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b49985-896d-40d4-b47e-c88f88fd3c4a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.406138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=0bef02839fb44788b776f6630efc17ac) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:53.406331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b49985-896d-40d4-b47e-c88f88fd3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.406423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:53.406678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b49985-896d-40d4-b47e-c88f88fd3c4a None admin] PATH_INFO: `/v3/users/0bef02839fb44788b776f6630efc17ac` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.407943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:53.408041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b49985-896d-40d4-b47e-c88f88fd3c4a None admin] RBAC: Authorizing `identity:get_user(user_id=0bef02839fb44788b776f6630efc17ac)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:53.409458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b49985-896d-40d4-b47e-c88f88fd3c4a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:53.411439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 92/197] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:53 2022] GET /identity/v3/users/0bef02839fb44788b776f6630efc17ac => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:53.412485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f7b9a-868f-4f66-a222-23a8fd75c5b7 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.412768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f7b9a-868f-4f66-a222-23a8fd75c5b7 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.413070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f7b9a-868f-4f66-a222-23a8fd75c5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.418994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 106/198] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:53 2022] GET /identity/v3/role_assignments?user.id=0bef02839fb44788b776f6630efc17ac&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:53.419885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.436378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-573f7b9a-868f-4f66-a222-23a8fd75c5b7 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:53.438068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-573f7b9a-868f-4f66-a222-23a8fd75c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:53.542283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 93/199] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:53.561551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1110b50-7583-4648-a2d9-61293101455e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.566689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f7b9a-868f-4f66-a222-23a8fd75c5b7 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.615138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.615793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.616057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.616326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.696652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f7b9a-868f-4f66-a222-23a8fd75c5b7 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.704291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:53.706057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:53.746033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:53.746923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:53.747205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:53.747477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:53.762875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:53.773536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:53.774525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:53.775546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 94/200] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:53.780870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.823690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 107/201] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:53.835306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:53.928477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa04629-a28d-47ac-8895-fd26d71255e8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:53.973185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ffbfe-17fd-4d47-91bd-35bf2f4bec38 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:54.010967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:54.011902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:54.012178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:54.012407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:54.012519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:54.013432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:54.013714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:54.014109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:54.014564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:54.015177 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:54.015724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:54.016222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:54.016928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 108/202] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:53 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:54.018171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:54.024860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:54.039389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 95/203] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:54.046949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:54.135154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a9177-830f-41f6-b3cb-6beb793dffed None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:54.172985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:54.173824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:54.174151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:54.174442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:54.175637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] RBAC: Adding query filter params (role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=0bef02839fb44788b776f6630efc17ac) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:54.175947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:54.177668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:54.184765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 109/204] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:54 2022] GET /identity/v3/role_assignments?user.id=0bef02839fb44788b776f6630efc17ac&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:54.270809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976285d1-d22b-40a3-8890-f5f9cc08c0ce None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:54.308824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:54.312161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:54.312161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:54.312161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] PATH_INFO: `/v3/users/294d56c712ed49e3b8acbd7739911193` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:54.312161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] RBAC: Authorizing `identity:get_user(user_id=294d56c712ed49e3b8acbd7739911193)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:54.313058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:54.314945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 96/205] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:54 2022] GET /identity/v3/users/294d56c712ed49e3b8acbd7739911193 => generated 289 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:54.320518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:54.428886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f70a-6546-4b75-9e56-6c3e888f5ba8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:54.467402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:54.468070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:54.468347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:54.468618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:54.543315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:54.544856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:54.645370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 110/206] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:54.651956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:54.756657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fd1dc-239c-42f5-8918-1f3f9f385589 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:54.791660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ab3e-3f8a-4ca8-b40b-5893da6a9d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:54.792546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ab3e-3f8a-4ca8-b40b-5893da6a9d59 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:54.792845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ab3e-3f8a-4ca8-b40b-5893da6a9d59 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:54.793126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ab3e-3f8a-4ca8-b40b-5893da6a9d59 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:54.794348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ab3e-3f8a-4ca8-b40b-5893da6a9d59 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:54.795818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ab3e-3f8a-4ca8-b40b-5893da6a9d59 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:54.797607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 97/207] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:54 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:54.804287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:54.913700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3895f-7bcf-4959-b806-c3fec4194c78 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:54.964539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f48849-77da-45dd-a578-df07430a74b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:54.965612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48849-77da-45dd-a578-df07430a74b2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:54.966094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48849-77da-45dd-a578-df07430a74b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:54.966595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48849-77da-45dd-a578-df07430a74b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:54.967853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f48849-77da-45dd-a578-df07430a74b2 None admin] RBAC: Adding query filter params (role.id=070af100905e4b06b408f229d26a3bdb, scope.project.id=c7555237bf4842dd8f28617418adec46, user.id=294d56c712ed49e3b8acbd7739911193) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:54.968407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f48849-77da-45dd-a578-df07430a74b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:54.970127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f48849-77da-45dd-a578-df07430a74b2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:54.981579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 111/208] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:54 2022] GET /identity/v3/role_assignments?user.id=294d56c712ed49e3b8acbd7739911193&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=070af100905e4b06b408f229d26a3bdb => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:56.029229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d789-abb8-4842-b8ff-41ccce37eb00 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.030273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d789-abb8-4842-b8ff-41ccce37eb00 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.030761 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d789-abb8-4842-b8ff-41ccce37eb00 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.032381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 98/209] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:56.038407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0e42-2eb2-4232-8a92-93d003df31c9 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.039249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0e42-2eb2-4232-8a92-93d003df31c9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.039743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0e42-2eb2-4232-8a92-93d003df31c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.057029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f18c0e42-2eb2-4232-8a92-93d003df31c9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:56.058546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f18c0e42-2eb2-4232-8a92-93d003df31c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:56.130038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 112/210] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:56.139471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd58663-85f3-4d90-bdca-2292ab7f8e51 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.140047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd58663-85f3-4d90-bdca-2292ab7f8e51 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.140515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd58663-85f3-4d90-bdca-2292ab7f8e51 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.142400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 99/211] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:56.150888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f28b4-43b3-416b-9414-26e58d9123c2 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.151446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f28b4-43b3-416b-9414-26e58d9123c2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.151981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f28b4-43b3-416b-9414-26e58d9123c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.156919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd58663-85f3-4d90-bdca-2292ab7f8e51 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.169907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-087f28b4-43b3-416b-9414-26e58d9123c2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:56.171409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-087f28b4-43b3-416b-9414-26e58d9123c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:56.250839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 113/212] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:56.271214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd58663-85f3-4d90-bdca-2292ab7f8e51 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.275829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f28b4-43b3-416b-9414-26e58d9123c2 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.312634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae233c-5efa-4671-804e-70c5a6fd94fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.313495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae233c-5efa-4671-804e-70c5a6fd94fd None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.314509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae233c-5efa-4671-804e-70c5a6fd94fd None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.314509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae233c-5efa-4671-804e-70c5a6fd94fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.328312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae233c-5efa-4671-804e-70c5a6fd94fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-00ae233c-5efa-4671-804e-70c5a6fd94fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:56.335955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:56.337257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:37:56.338141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 100/213] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:56.339193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346f1ae-4bd2-48c5-8a30-c9b33f31640b None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.339193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346f1ae-4bd2-48c5-8a30-c9b33f31640b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.339193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346f1ae-4bd2-48c5-8a30-c9b33f31640b None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.340454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 101/214] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:56.345653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe63f4-46db-41f6-82b6-a2823179f6c0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.345937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe63f4-46db-41f6-82b6-a2823179f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.346213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe63f4-46db-41f6-82b6-a2823179f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.362400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-28fe63f4-46db-41f6-82b6-a2823179f6c0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:56.363565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-28fe63f4-46db-41f6-82b6-a2823179f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:56.384740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f28b4-43b3-416b-9414-26e58d9123c2 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.419848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.420663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.420994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.421280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.427118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 102/215] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:56.432230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe63f4-46db-41f6-82b6-a2823179f6c0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.433290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:56.440992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:56.441906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:37:56.442699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:56.446995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 114/216] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:37:56.448400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.536402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe63f4-46db-41f6-82b6-a2823179f6c0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.547842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e790f-8969-4918-9b7f-559f3409f4ef None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.577460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.578185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.578631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.578920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.580349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:56.580776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:56.582257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:56.584009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.584720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.585025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.585304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.586093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] RBAC: Adding query filter params (name=member) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:56.586397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:56.587845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:56.593220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 103/217] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:56.594974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.602242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 115/218] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:56.602242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.703386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cecd2-b4ba-45bc-ba29-dfd31c33e6a0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.704395 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853d483-2d80-43da-af05-b894589fff9e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.736421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.737261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.737538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.737809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.738123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.738973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:56.739083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.739285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.739557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] PATH_INFO: `/v3/users/0bef02839fb44788b776f6630efc17ac` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.740410 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:56.740821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] RBAC: Authorizing `identity:get_user(user_id=0bef02839fb44788b776f6630efc17ac)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:56.742239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 104/219] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:56.742340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:56.748608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 116/220] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/users/0bef02839fb44788b776f6630efc17ac => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:56.748786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.749360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.844293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ecbd5-26a6-4064-ace8-964cdd127be6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.848441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1cdfd-1151-4502-aa60-af7a049e5e82 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:56.877859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.878772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.879047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.879320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] PATH_INFO: `/v3/users/f49e208b7ece4d1aa0563873eef6c60e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.883485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eeba10-fd44-49df-8c15-d9f06d5c8170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:56.884164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeba10-fd44-49df-8c15-d9f06d5c8170 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.884453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeba10-fd44-49df-8c15-d9f06d5c8170 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.884727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eeba10-fd44-49df-8c15-d9f06d5c8170 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.885565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eeba10-fd44-49df-8c15-d9f06d5c8170 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:56.887019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eeba10-fd44-49df-8c15-d9f06d5c8170 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:56.892494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=f49e208b7ece4d1aa0563873eef6c60e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:56.893965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:56.895718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 105/221] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/users/f49e208b7ece4d1aa0563873eef6c60e => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:56.896469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:56.913658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 117/222] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:37:56 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:37:56.915037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d847-c972-4d36-a757-41f3b7cbe442 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:56.915323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d847-c972-4d36-a757-41f3b7cbe442 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:56.915597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6d847-c972-4d36-a757-41f3b7cbe442 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:56.916948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 118/223] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:56.917688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6d847-c972-4d36-a757-41f3b7cbe442 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:57.011801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0a41e-51fa-49a2-9499-82354503f1f7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.045832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6d847-c972-4d36-a757-41f3b7cbe442 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.050817 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46317f4-828c-4b34-8dbc-fcb5e423881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.051734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46317f4-828c-4b34-8dbc-fcb5e423881c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.052015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46317f4-828c-4b34-8dbc-fcb5e423881c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.052290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46317f4-828c-4b34-8dbc-fcb5e423881c None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.053703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46317f4-828c-4b34-8dbc-fcb5e423881c None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:57.055195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46317f4-828c-4b34-8dbc-fcb5e423881c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:57.057363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 106/224] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:57.058540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b1a91-ebbb-46f5-b7d9-4c773d827d7d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.058837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b1a91-ebbb-46f5-b7d9-4c773d827d7d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.059110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b1a91-ebbb-46f5-b7d9-4c773d827d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.076647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e89b1a91-ebbb-46f5-b7d9-4c773d827d7d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:57.077929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e89b1a91-ebbb-46f5-b7d9-4c773d827d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:57.086342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.087030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.087311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.087588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.148464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 107/225] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:37:57.149753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b1a91-ebbb-46f5-b7d9-4c773d827d7d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:57.159538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:57.161129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:57.271459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b1a91-ebbb-46f5-b7d9-4c773d827d7d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.287237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 119/226] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:37:57.288398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:57.315168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.316182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.316530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.316906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/0bef02839fb44788b776f6630efc17ac/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.321926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] RBAC: Authorizing `identity:create_grant(project_id=cefff7c16392441d88d3526f6461ccb4, user_id=0bef02839fb44788b776f6630efc17ac, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:57.323970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:57.342512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 108/227] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:37:57 2022] PUT /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/0bef02839fb44788b776f6630efc17ac/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:57.343661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:57.473111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfd723-7eb8-4f13-9416-ec63cd746471 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.535959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.539356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.539982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.540527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.542962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b356062-c19d-4828-8bf5-3090e97a3298 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.556123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:37:57.566093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:37:57.567154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:37:57.567943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:37:57.570599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 120/228] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:37:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:37:57.577683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:57.611296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.612196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.612448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.612731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.614021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:57.615507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:57.617639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 109/229] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:57 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:57.623977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:57.690512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d2654-f201-42a2-ae35-99dd99924a4d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.726772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.727393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.727673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.727943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.728712 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:57.731946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:57.733580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:57.742727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 121/230] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:37:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:57.748361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:57.768899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1145e2-9a11-4827-97fc-2a045afc1a0c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.837354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.838230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.838553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.838831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.839688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=f49e208b7ece4d1aa0563873eef6c60e) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:37:57.840000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:57.844817 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:57.848995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 110/231] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:37:57 2022] GET /identity/v3/role_assignments?user.id=f49e208b7ece4d1aa0563873eef6c60e&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=4e8a255633994167b69cee0662997d5e => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:57.906350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca021b-c4a0-4ec6-9f84-5a0766df3a28 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:57.948573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:57.949705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:57.950090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:57.950482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] PATH_INFO: `/v3/users/0bef02839fb44788b776f6630efc17ac` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:57.951892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] RBAC: Authorizing `identity:get_user(user_id=0bef02839fb44788b776f6630efc17ac)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:57.953514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:57.955357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 122/232] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:37:57 2022] GET /identity/v3/users/0bef02839fb44788b776f6630efc17ac => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:37:57.961455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:58.087856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2cfb0-a79c-47db-8ab8-e967db22dd69 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:58.124135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c11aec-74d9-486b-9a2d-b4d1664d6925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:58.125035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c11aec-74d9-486b-9a2d-b4d1664d6925 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:58.125316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c11aec-74d9-486b-9a2d-b4d1664d6925 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:58.125586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c11aec-74d9-486b-9a2d-b4d1664d6925 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:58.126801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c11aec-74d9-486b-9a2d-b4d1664d6925 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:58.128309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c11aec-74d9-486b-9a2d-b4d1664d6925 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:58.130190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 111/233] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:37:57 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:37:58.136426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:37:58.243180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704211a-a4ac-4894-9844-5fdfc0062c30 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:37:58.287118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b5414-bb0e-40c9-b84a-55cac66b3f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:37:58.288039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b5414-bb0e-40c9-b84a-55cac66b3f1a None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:58.288316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b5414-bb0e-40c9-b84a-55cac66b3f1a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:58.288588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b5414-bb0e-40c9-b84a-55cac66b3f1a None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/0bef02839fb44788b776f6630efc17ac/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:58.291036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b5414-bb0e-40c9-b84a-55cac66b3f1a None admin] RBAC: Authorizing `identity:create_grant(project_id=cefff7c16392441d88d3526f6461ccb4, user_id=0bef02839fb44788b776f6630efc17ac, role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:37:58.292572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b5414-bb0e-40c9-b84a-55cac66b3f1a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:37:58.312325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 123/234] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:37:58 2022] PUT /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/0bef02839fb44788b776f6630efc17ac/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:37:59.866697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d3871-03ea-488d-9a7e-5fd3f5be4729 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:59.867713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d3871-03ea-488d-9a7e-5fd3f5be4729 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:59.868214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d3871-03ea-488d-9a7e-5fd3f5be4729 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:59.870246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 112/235] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:37:59.875952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2e2dc-d36b-499c-a6b3-823b37655591 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:59.876818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2e2dc-d36b-499c-a6b3-823b37655591 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:59.877285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2e2dc-d36b-499c-a6b3-823b37655591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:59.886751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e50312-86e7-426e-be17-6a6563f034f4 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:59.887076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e50312-86e7-426e-be17-6a6563f034f4 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:59.887286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e50312-86e7-426e-be17-6a6563f034f4 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:59.889185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 113/236] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:37:59.898618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26def2d6-5bb8-4658-993e-af8041e02002 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:37:59.898966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26def2d6-5bb8-4658-993e-af8041e02002 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:37:59.899179 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26def2d6-5bb8-4658-993e-af8041e02002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:37:59.902176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-41b2e2dc-d36b-499c-a6b3-823b37655591 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:59.903638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-41b2e2dc-d36b-499c-a6b3-823b37655591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:59.920372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-26def2d6-5bb8-4658-993e-af8041e02002 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:37:59.929816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-26def2d6-5bb8-4658-993e-af8041e02002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:37:59.977244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 124/237] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:00.002121 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2e2dc-d36b-499c-a6b3-823b37655591 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.043620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 114/238] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:00.080308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26def2d6-5bb8-4658-993e-af8041e02002 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.102850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2e2dc-d36b-499c-a6b3-823b37655591 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.146891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:00.148415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.148946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.149404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.151015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:00.152778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:00.154994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 125/239] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:00.160698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.209948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26def2d6-5bb8-4658-993e-af8041e02002 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.244371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ea146-7872-4f80-bc3d-089da7c8eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:00.245254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ea146-7872-4f80-bc3d-089da7c8eca7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.245535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ea146-7872-4f80-bc3d-089da7c8eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.245809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ea146-7872-4f80-bc3d-089da7c8eca7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.256497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ea146-7872-4f80-bc3d-089da7c8eca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:00.262446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659a76-7388-4e9b-a5c3-bb6e2f5e5fb2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-882ea146-7872-4f80-bc3d-089da7c8eca7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:00.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:00.265293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:00.266092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 115/240] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:00.267491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c641e6-7002-4ae6-91db-7d4de0544a42 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.267491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c641e6-7002-4ae6-91db-7d4de0544a42 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.267662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c641e6-7002-4ae6-91db-7d4de0544a42 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.269714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 116/241] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:00.277780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70760-cdfc-4d7a-bfb5-1edf1df61cef None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.277780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70760-cdfc-4d7a-bfb5-1edf1df61cef None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.277780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70760-cdfc-4d7a-bfb5-1edf1df61cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.295764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-78e70760-cdfc-4d7a-bfb5-1edf1df61cef None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:00.296823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-78e70760-cdfc-4d7a-bfb5-1edf1df61cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:00.306508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:00.307619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.308049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.308438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.309355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:00.311057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:00.345269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 126/242] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:00 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:38:00.346316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.383005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 117/243] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:00.407961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e70760-cdfc-4d7a-bfb5-1edf1df61cef None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.474469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e1811-106b-4e19-81a5-84caca3cd409 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.511381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:00.512073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.512362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.512641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.513454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:00.513764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:00.515220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:00.527903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 127/244] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:00.533931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.534758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e70760-cdfc-4d7a-bfb5-1edf1df61cef None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.580095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:00.581028 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.581317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.584111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.611850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:00.625420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:00.627563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:00.629515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 118/245] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:00.640554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.726822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e3c6-0651-44d2-93d6-ece24621159f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.772775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:00.774051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.774340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.774783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] PATH_INFO: `/v3/users/d6b08fd00cca4866907fce4c3517de7f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.794273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e8ebd-c6d2-4770-9893-6f18698de37f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.794617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b08fd00cca4866907fce4c3517de7f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:00.796337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:00.799408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 128/246] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/users/d6b08fd00cca4866907fce4c3517de7f => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:00.808395 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:00.839223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6c1ed-b76a-4fe5-b441-7bf785a8e875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:00.839921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6c1ed-b76a-4fe5-b441-7bf785a8e875 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.840211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6c1ed-b76a-4fe5-b441-7bf785a8e875 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.840483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6c1ed-b76a-4fe5-b441-7bf785a8e875 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.841361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6c1ed-b76a-4fe5-b441-7bf785a8e875 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:00.841671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6c1ed-b76a-4fe5-b441-7bf785a8e875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:00.844881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6c1ed-b76a-4fe5-b441-7bf785a8e875 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:00.854165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 119/247] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:00.855426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a46224-03a5-4a95-b553-066affc9a6d6 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.855710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a46224-03a5-4a95-b553-066affc9a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.856037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a46224-03a5-4a95-b553-066affc9a6d6 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.857644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 120/248] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:00.863248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab3655-99b3-46d2-9c00-16a497d6e108 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:00.863530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab3655-99b3-46d2-9c00-16a497d6e108 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:00.863805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab3655-99b3-46d2-9c00-16a497d6e108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:00.880629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0aab3655-99b3-46d2-9c00-16a497d6e108 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:00.881880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0aab3655-99b3-46d2-9c00-16a497d6e108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:00.947090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c61c8d-3934-4edb-bbe4-eff180957588 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:00.958005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 121/249] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:00.958945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab3655-99b3-46d2-9c00-16a497d6e108 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.007111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.008028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.008152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.008330 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.086223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:01.087736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:01.109070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab3655-99b3-46d2-9c00-16a497d6e108 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:01.176415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.177625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.178073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.178344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] PATH_INFO: `/v3/users/0bef02839fb44788b776f6630efc17ac` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.179691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] RBAC: Authorizing `identity:get_user(user_id=0bef02839fb44788b776f6630efc17ac)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:01.181242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:01.183425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 122/250] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/users/0bef02839fb44788b776f6630efc17ac => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:01.189434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.281860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 129/251] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:38:01.282739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.294583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa656534-ed48-4527-9c10-2331aa94a0e6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:01.363496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.364307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.364587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.364880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.385899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:01.393645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:01.394579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:01.395417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 123/252] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:01.397040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.455658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e87afb-20c9-4ad4-a8b5-a4fb1e7bf264 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:01.488679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ff58f-116a-4c11-a502-5b5d253ca781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.489507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff58f-116a-4c11-a502-5b5d253ca781 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.489823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff58f-116a-4c11-a502-5b5d253ca781 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.490093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff58f-116a-4c11-a502-5b5d253ca781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.535894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a56ff-f8bf-4c49-ad11-1d838be81af3 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:01.570670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.571715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.572646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.573001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.574436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ff58f-116a-4c11-a502-5b5d253ca781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:01.574628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:01.575938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ff58f-116a-4c11-a502-5b5d253ca781 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:01.576138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:01.578313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 124/253] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:01 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:01.579809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.697209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91225098-b9f7-4a57-8d78-b8004327970f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:01.703788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 130/254] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:01.706165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eada92-9536-462d-9c6c-def04d58ab9b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.707049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eada92-9536-462d-9c6c-def04d58ab9b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.707529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eada92-9536-462d-9c6c-def04d58ab9b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.709829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 131/255] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:01.710894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eada92-9536-462d-9c6c-def04d58ab9b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.738276 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.739387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.739387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.739555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.740287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:01.740587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:01.742031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:01.751499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 125/256] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:01.752332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.848681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eada92-9536-462d-9c6c-def04d58ab9b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:01.866569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346be54e-7a5c-4ec8-b7dd-5ee7d5555e2c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:01.891416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecaf73-ded5-4106-b2db-3b4190997205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.892470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecaf73-ded5-4106-b2db-3b4190997205 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.892981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecaf73-ded5-4106-b2db-3b4190997205 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.893509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecaf73-ded5-4106-b2db-3b4190997205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.894571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecaf73-ded5-4106-b2db-3b4190997205 None admin] RBAC: Adding query filter params (role.id=070af100905e4b06b408f229d26a3bdb, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=d6b08fd00cca4866907fce4c3517de7f) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:01.895076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecaf73-ded5-4106-b2db-3b4190997205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:01.896754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecaf73-ded5-4106-b2db-3b4190997205 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:01.902244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:01.903123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.903405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.903675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.904750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 132/257] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:01 2022] GET /identity/v3/role_assignments?user.id=d6b08fd00cca4866907fce4c3517de7f&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=070af100905e4b06b408f229d26a3bdb => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:01.905281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:01.906726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6b333-f887-4633-9726-bba4a5bd456e None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:01.906933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:01.907411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6b333-f887-4633-9726-bba4a5bd456e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:01.907897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6b333-f887-4633-9726-bba4a5bd456e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:01.908992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 126/258] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:01 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:01.909964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:01.926883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-49f6b333-f887-4633-9726-bba4a5bd456e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:01.928584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-49f6b333-f887-4633-9726-bba4a5bd456e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:02.009495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 133/259] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:02.011331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6b333-f887-4633-9726-bba4a5bd456e None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:02.022911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553473-d7d9-424b-ae3c-26bf52da2c1a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.059819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.060770 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.061082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.061367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] PATH_INFO: `/v3/users/f49e208b7ece4d1aa0563873eef6c60e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.062685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=f49e208b7ece4d1aa0563873eef6c60e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:02.064292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:02.066575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 127/260] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:01 2022] GET /identity/v3/users/f49e208b7ece4d1aa0563873eef6c60e => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:02.067289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:02.118487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6b333-f887-4633-9726-bba4a5bd456e None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.156106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.157290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.157783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.158260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.159344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=0bef02839fb44788b776f6630efc17ac) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:02.159880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:02.161702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:02.169021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 134/261] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:02 2022] GET /identity/v3/role_assignments?user.id=0bef02839fb44788b776f6630efc17ac&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:02.170279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:02.181344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eddcc7-93e3-44aa-bb7b-c6201fbc4fe9 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.223738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.224603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.224911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.225195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.245067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:02.260169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:02.261207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:02.262448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 128/262] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:02.267694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:02.286679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e29081-45a1-465f-a4a4-02ae4a7edc8c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.340619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.341556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.341832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.342098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.344680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:02.346414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:02.349310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 135/263] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:02 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:02.355588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:02.399185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db19a7c-9c83-4616-8ffb-8b55501a1a9b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.434531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.435673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.436154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.436622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.437624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:02.438222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:02.439854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:02.449324 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 129/264] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:02.454813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:02.457851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56f18-fef2-45a6-88d9-46d49c9c4334 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.502554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.503482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.503760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.504948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/f49e208b7ece4d1aa0563873eef6c60e/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.510310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] RBAC: Authorizing `identity:create_grant(project_id=cefff7c16392441d88d3526f6461ccb4, user_id=f49e208b7ece4d1aa0563873eef6c60e, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:02.511816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:02.539727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 136/265] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:38:02 2022] PUT /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/f49e208b7ece4d1aa0563873eef6c60e/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:02.580743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3fc9a-a21f-45b2-a4df-6e23390e40a9 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.618536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.619975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.620481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.621016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] PATH_INFO: `/v3/users/0bef02839fb44788b776f6630efc17ac` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.622972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] RBAC: Authorizing `identity:get_user(user_id=0bef02839fb44788b776f6630efc17ac)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:02.624757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:02.627034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 130/266] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:02 2022] GET /identity/v3/users/0bef02839fb44788b776f6630efc17ac => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:02.633232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:02.779234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c868d-2f95-4d92-ae92-ca262c208066 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:02.813913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:02.814597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:02.814855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:02.815124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:02.889029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:02.890534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:02.997336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 137/267] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:03.005298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:03.146948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45299b7-6063-446c-95d0-0e327138715c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:03.191697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2eb59-7a73-464c-a39f-536e1d84379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:03.192995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2eb59-7a73-464c-a39f-536e1d84379a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:03.193483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2eb59-7a73-464c-a39f-536e1d84379a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:03.193980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2eb59-7a73-464c-a39f-536e1d84379a None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:03.195418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2eb59-7a73-464c-a39f-536e1d84379a None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:03.197079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2eb59-7a73-464c-a39f-536e1d84379a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:03.199299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 131/268] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:03 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:03.205559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:03.383011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c53d5b-3fbc-4546-96cb-413e9b0d9f73 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:03.432605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47fa87-1cf6-4f85-a2d3-8d44f7aa9edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:03.433221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47fa87-1cf6-4f85-a2d3-8d44f7aa9edf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:03.433498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47fa87-1cf6-4f85-a2d3-8d44f7aa9edf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:03.433768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47fa87-1cf6-4f85-a2d3-8d44f7aa9edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:03.434612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47fa87-1cf6-4f85-a2d3-8d44f7aa9edf None admin] RBAC: Adding query filter params (role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=0bef02839fb44788b776f6630efc17ac) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:03.434929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47fa87-1cf6-4f85-a2d3-8d44f7aa9edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:03.438167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47fa87-1cf6-4f85-a2d3-8d44f7aa9edf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:03.447205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 138/269] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:03 2022] GET /identity/v3/role_assignments?user.id=0bef02839fb44788b776f6630efc17ac&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:03.688100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d376-1de7-4b32-bf87-059998a718ed None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:03.688366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d376-1de7-4b32-bf87-059998a718ed None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:03.688593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d376-1de7-4b32-bf87-059998a718ed None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:03.690390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 132/270] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:03.696080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a34e44-bd6d-4595-bd01-9ba9f3b6dd7b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:03.696403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a34e44-bd6d-4595-bd01-9ba9f3b6dd7b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:03.696645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a34e44-bd6d-4595-bd01-9ba9f3b6dd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:03.726107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a0a34e44-bd6d-4595-bd01-9ba9f3b6dd7b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:03.727369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a0a34e44-bd6d-4595-bd01-9ba9f3b6dd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:03.807286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 139/271] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:03.831800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165d376-1de7-4b32-bf87-059998a718ed None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:03.874209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b273b-8704-4386-a68e-115ff3fc4f3b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:03.874677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b273b-8704-4386-a68e-115ff3fc4f3b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:03.875064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b273b-8704-4386-a68e-115ff3fc4f3b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:03.876583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 140/272] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:03.882472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ee696-2178-4b7e-b225-0d050b35adcb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:03.882841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ee696-2178-4b7e-b225-0d050b35adcb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:03.883061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ee696-2178-4b7e-b225-0d050b35adcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:03.907966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1a7ee696-2178-4b7e-b225-0d050b35adcb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:03.909194 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1a7ee696-2178-4b7e-b225-0d050b35adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:03.976540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165d376-1de7-4b32-bf87-059998a718ed None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:03.982012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 141/273] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:04.007121 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ee696-2178-4b7e-b225-0d050b35adcb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:04.025565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:04.026427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:04.026711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:04.026984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:04.041614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:04.049635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:04.050753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:04.051571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:04.054705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 133/274] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:04.057423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:04.110713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ee696-2178-4b7e-b225-0d050b35adcb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:04.144514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:04.145377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:04.145661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:04.145938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:04.147144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:04.149027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:04.150799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 142/275] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:04.156365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:04.158473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9c581-89e5-4e85-94e6-9e2855bd5494 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:04.192675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:04.193440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:04.193823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:04.194103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:04.194870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:04.195187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:04.196577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:04.210350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 134/276] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:04.211810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:04.257137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3082-10ce-4fff-8fe4-5ddafcaa7fdc None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:04.292256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:04.292994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:04.293277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:04.293555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:04.294283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:04.295741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:04.316756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e38f52-73b1-48d2-b8c1-f1664243b70b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:04.328928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 143/277] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:04 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:38:04.357399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1078490-8cda-4893-b7a2-e3491f24a0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:04.358357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078490-8cda-4893-b7a2-e3491f24a0db None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:04.358645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078490-8cda-4893-b7a2-e3491f24a0db None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:04.358955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1078490-8cda-4893-b7a2-e3491f24a0db None admin] PATH_INFO: `/v3/users/5056e3c0d51e41bbafd3e6a813c0719e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:04.376142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1078490-8cda-4893-b7a2-e3491f24a0db None admin] RBAC: Authorizing `identity:get_user(user_id=5056e3c0d51e41bbafd3e6a813c0719e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:04.377760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1078490-8cda-4893-b7a2-e3491f24a0db None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:04.384769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 135/278] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:04 2022] GET /identity/v3/users/5056e3c0d51e41bbafd3e6a813c0719e => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:04.386250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:04.502015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacbef2-040e-4e85-8efd-46c818b46b95 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:04.540549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50e1da-4871-4371-99c1-2c806a985000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:04.541645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e1da-4871-4371-99c1-2c806a985000 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:04.542087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e1da-4871-4371-99c1-2c806a985000 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:04.542512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e1da-4871-4371-99c1-2c806a985000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:04.543511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50e1da-4871-4371-99c1-2c806a985000 None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, user.id=5056e3c0d51e41bbafd3e6a813c0719e, scope.system=all) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:04.543981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50e1da-4871-4371-99c1-2c806a985000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:04.545621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50e1da-4871-4371-99c1-2c806a985000 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:04.551920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 144/279] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 4 14:38:04 2022] GET /identity/v3/role_assignments?user.id=5056e3c0d51e41bbafd3e6a813c0719e&scope.system=all&role.id=4e8a255633994167b69cee0662997d5e => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:05.273949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687b17-8eb4-4c0a-82e5-f33d065d31b6 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:05.274518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687b17-8eb4-4c0a-82e5-f33d065d31b6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:05.274518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687b17-8eb4-4c0a-82e5-f33d065d31b6 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:05.276269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 136/280] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:05.281722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07154b-c56c-4c60-8ec0-4473cdb12a30 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:05.282141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07154b-c56c-4c60-8ec0-4473cdb12a30 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:05.282310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07154b-c56c-4c60-8ec0-4473cdb12a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:05.309873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ae07154b-c56c-4c60-8ec0-4473cdb12a30 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:05.311097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ae07154b-c56c-4c60-8ec0-4473cdb12a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:05.407177 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 145/281] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:05.441961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687b17-8eb4-4c0a-82e5-f33d065d31b6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:05.574331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687b17-8eb4-4c0a-82e5-f33d065d31b6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:05.614431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c4fb8-c51e-4969-8a44-33dd0052ea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:05.615253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4fb8-c51e-4969-8a44-33dd0052ea65 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:05.615535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4fb8-c51e-4969-8a44-33dd0052ea65 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:05.615811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4fb8-c51e-4969-8a44-33dd0052ea65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:05.630283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c4fb8-c51e-4969-8a44-33dd0052ea65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-bf7c4fb8-c51e-4969-8a44-33dd0052ea65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:05.638206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:05.639145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:05.639954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 137/282] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:05.650294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07154b-c56c-4c60-8ec0-4473cdb12a30 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:05.745488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b204-bbff-4276-9c8e-61e5da6cecac None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:05.746185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b204-bbff-4276-9c8e-61e5da6cecac None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:05.746666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130b204-bbff-4276-9c8e-61e5da6cecac None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:05.748272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 138/283] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:05.755685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad5dd-c90e-4840-8e94-077b3449f962 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:05.755685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad5dd-c90e-4840-8e94-077b3449f962 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:05.755685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad5dd-c90e-4840-8e94-077b3449f962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:05.779309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4daad5dd-c90e-4840-8e94-077b3449f962 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:05.780873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4daad5dd-c90e-4840-8e94-077b3449f962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:05.793759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07154b-c56c-4c60-8ec0-4473cdb12a30 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:05.835000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:05.836051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:05.836536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:05.837030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:05.838065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:05.838582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:05.840208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:05.849674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 146/284] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:05.855525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:05.922583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 139/285] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:05.954639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad5dd-c90e-4840-8e94-077b3449f962 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:06.023608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5caa32-adac-4378-88fe-6a97adf9a2f7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:06.082081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:06.083356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:06.083825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:06.084951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] PATH_INFO: `/v3/users/d6b08fd00cca4866907fce4c3517de7f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:06.086538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b08fd00cca4866907fce4c3517de7f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:06.089588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:06.092201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 147/286] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:05 2022] GET /identity/v3/users/d6b08fd00cca4866907fce4c3517de7f => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:06.101990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:06.134736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad5dd-c90e-4840-8e94-077b3449f962 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:06.194573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:06.195750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:06.196235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:06.196686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:06.213373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:06.221947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:06.222939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:06.223735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:06.233074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 140/287] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:06.238879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:06.282738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72edd4-1168-41f0-aca4-6e16c6de10a6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:06.360603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:06.361860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:06.362344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:06.362812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:06.364933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:06.366605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:06.368694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 148/288] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:06 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:06.377846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:06.405663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff471b8-9f20-4897-97bc-58d0345ada60 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:06.450901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:06.452125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:06.452710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:06.453241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:06.454813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:06.456281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:06.458292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:06.470816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 141/289] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:06.476100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:06.587586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ee486-8bef-460f-b5c4-91f39a0c5e67 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:06.619358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f676b22-9c51-4b96-9750-bd5bc985c229 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:06.657433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:06.658772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:06.659243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:06.659716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/d6b08fd00cca4866907fce4c3517de7f/roles/070af100905e4b06b408f229d26a3bdb` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:06.663312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=cefff7c16392441d88d3526f6461ccb4, user_id=d6b08fd00cca4866907fce4c3517de7f, role_id=070af100905e4b06b408f229d26a3bdb)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:06.665715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:06.687272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7353d5-4ea9-42af-a18f-6eacac2451e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:06.688553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7353d5-4ea9-42af-a18f-6eacac2451e5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:06.689795 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 149/290] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:38:06 2022] PUT /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4/users/d6b08fd00cca4866907fce4c3517de7f/roles/070af100905e4b06b408f229d26a3bdb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:06.690289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7353d5-4ea9-42af-a18f-6eacac2451e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:06.690762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7353d5-4ea9-42af-a18f-6eacac2451e5 None admin] PATH_INFO: `/v3/users/f49e208b7ece4d1aa0563873eef6c60e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:06.692260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7353d5-4ea9-42af-a18f-6eacac2451e5 None admin] RBAC: Authorizing `identity:get_user(user_id=f49e208b7ece4d1aa0563873eef6c60e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:06.693980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7353d5-4ea9-42af-a18f-6eacac2451e5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:06.696399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 142/291] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:06 2022] GET /identity/v3/users/f49e208b7ece4d1aa0563873eef6c60e => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:06.705730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:06.854861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92256f-9805-4423-86e6-56eacd19d2b3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:06.906742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:06.907790 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:06.908293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:06.908760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:06.988753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:06.990655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:07.070886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c08f-0f20-4a65-adc1-5f79a3a8f8b5 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.071229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c08f-0f20-4a65-adc1-5f79a3a8f8b5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.071441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c08f-0f20-4a65-adc1-5f79a3a8f8b5 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.072971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 143/292] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:07.078277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245c9b-8abc-4848-8292-36c73a4725fd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.078561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245c9b-8abc-4848-8292-36c73a4725fd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.078840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245c9b-8abc-4848-8292-36c73a4725fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.101413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 150/293] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:07.110443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-18245c9b-8abc-4848-8292-36c73a4725fd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:07.111720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-18245c9b-8abc-4848-8292-36c73a4725fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:07.111966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.198413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 144/294] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:07.227893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18245c9b-8abc-4848-8292-36c73a4725fd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.277499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fc85a-60be-4dd4-b8af-28844562f9d0 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:07.330343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:07.331254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.331533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.331995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.333384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:07.334897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:07.336696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 151/295] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:07 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:07.355402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.366487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18245c9b-8abc-4848-8292-36c73a4725fd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:07.409015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423ad85-f739-4653-8594-398d8cf5795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:07.411332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ad85-f739-4653-8594-398d8cf5795c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.411797 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ad85-f739-4653-8594-398d8cf5795c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.412213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423ad85-f739-4653-8594-398d8cf5795c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.426054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423ad85-f739-4653-8594-398d8cf5795c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-3423ad85-f739-4653-8594-398d8cf5795c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:07.436414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:07.437563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:07.438559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:07.441745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 145/296] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:07.443207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c191-a2d3-4a9e-91c2-8a7baefe77c2 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.443648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c191-a2d3-4a9e-91c2-8a7baefe77c2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.444241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393c191-a2d3-4a9e-91c2-8a7baefe77c2 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.446397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 146/297] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:07.451382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393c191-a2d3-4a9e-91c2-8a7baefe77c2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.495133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9afbad-61d8-4f25-b953-2226a525ccad None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:07.533050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78af65-73a4-4a6c-a603-7686afbcf8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:07.533731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78af65-73a4-4a6c-a603-7686afbcf8ac None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.534014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78af65-73a4-4a6c-a603-7686afbcf8ac None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.534293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78af65-73a4-4a6c-a603-7686afbcf8ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.535158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78af65-73a4-4a6c-a603-7686afbcf8ac None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=f49e208b7ece4d1aa0563873eef6c60e) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:07.535456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78af65-73a4-4a6c-a603-7686afbcf8ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:07.536957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78af65-73a4-4a6c-a603-7686afbcf8ac None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:07.543018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 152/298] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:07 2022] GET /identity/v3/role_assignments?user.id=f49e208b7ece4d1aa0563873eef6c60e&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=4e8a255633994167b69cee0662997d5e => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:07.544270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6f92-2630-4a1f-9b26-faf38dec0898 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.544556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6f92-2630-4a1f-9b26-faf38dec0898 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.544856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6f92-2630-4a1f-9b26-faf38dec0898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.567392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-bf9d6f92-2630-4a1f-9b26-faf38dec0898 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:07.569408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bf9d6f92-2630-4a1f-9b26-faf38dec0898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:07.583136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393c191-a2d3-4a9e-91c2-8a7baefe77c2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:07.629321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83fee0-b2e2-47e9-9ba7-5e44f429d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:07.630582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83fee0-b2e2-47e9-9ba7-5e44f429d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.631021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83fee0-b2e2-47e9-9ba7-5e44f429d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.631440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83fee0-b2e2-47e9-9ba7-5e44f429d6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.633038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83fee0-b2e2-47e9-9ba7-5e44f429d6f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:07.633584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83fee0-b2e2-47e9-9ba7-5e44f429d6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:07.635298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83fee0-b2e2-47e9-9ba7-5e44f429d6f2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:07.654535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 147/299] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:07.654535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc4db4-6394-49da-acb8-a12cbaa50c1a None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.654535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc4db4-6394-49da-acb8-a12cbaa50c1a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.654535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc4db4-6394-49da-acb8-a12cbaa50c1a None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.657054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 148/300] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:07.658395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc4db4-6394-49da-acb8-a12cbaa50c1a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.668914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 153/301] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:07.668914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6053f78-193b-4346-b3fc-e99bc17a7ba5 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.668914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6053f78-193b-4346-b3fc-e99bc17a7ba5 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.668914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6053f78-193b-4346-b3fc-e99bc17a7ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.687871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e6053f78-193b-4346-b3fc-e99bc17a7ba5 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:07.689706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e6053f78-193b-4346-b3fc-e99bc17a7ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:07.776084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 154/302] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:07.778547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6053f78-193b-4346-b3fc-e99bc17a7ba5 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.798404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc4db4-6394-49da-acb8-a12cbaa50c1a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:07.840884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:07.841787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.842064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.842341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] PATH_INFO: `/v3/users/5056e3c0d51e41bbafd3e6a813c0719e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.845045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] RBAC: Authorizing `identity:get_user(user_id=5056e3c0d51e41bbafd3e6a813c0719e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:07.845781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:07.847645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 149/303] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:07 2022] GET /identity/v3/users/5056e3c0d51e41bbafd3e6a813c0719e => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:07.849215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.904714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6053f78-193b-4346-b3fc-e99bc17a7ba5 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:07.943605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:07.944417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:07.944699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:07.945013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:07.956087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:07.964000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:07.965159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:07.966124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:07.970278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 155/304] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:07.971126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:07.980879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713d3ee-ed84-48a0-ab83-6cdbf907ce1a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:08.014364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:08.015182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:08.015465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:08.015732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:08.017095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:08.018533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:08.020402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 150/305] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:08.025020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:08.076955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db096875-ae7b-4ecc-9c48-98d8c657bd77 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:08.111628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:08.112519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:08.112806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:08.113218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] PATH_INFO: `/v3/system/users/5056e3c0d51e41bbafd3e6a813c0719e/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:08.114708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5056e3c0d51e41bbafd3e6a813c0719e, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:08.116112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:08.125953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b94648-f84a-48b5-8db0-087ba450b77e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:08.127369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 156/306] 10.222.0.32 () {58 vars in 1334 bytes} [Wed May 4 14:38:07 2022] PUT /identity/v3/system/users/5056e3c0d51e41bbafd3e6a813c0719e/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:08.128226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:08.172446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:08.173929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:08.174393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:08.174930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:08.176053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] RBAC: Adding query filter params (name=reader) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:08.176583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:08.178549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:08.187951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 151/307] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:08.193956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:08.258816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7939c6c-9b25-47a7-bdf3-f7624ccc73d8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:08.296025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:08.296720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:08.297051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:08.297527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] PATH_INFO: `/v3/groups` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:08.298654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:08.299899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:08.324647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 157/308] 10.222.0.32 () {60 vars in 1134 bytes} [Wed May 4 14:38:08 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:08.334010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93df99-f5be-4bd9-ba82-cd2920bd8fec None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:08.375092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e789ff-bab8-4686-b3a4-a6af61d4d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:08.376073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e789ff-bab8-4686-b3a4-a6af61d4d09f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:08.376363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e789ff-bab8-4686-b3a4-a6af61d4d09f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:08.376637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e789ff-bab8-4686-b3a4-a6af61d4d09f None admin] PATH_INFO: `/v3/users/3d92582157634b598abe103256a001ef` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:08.393571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e789ff-bab8-4686-b3a4-a6af61d4d09f None admin] RBAC: Authorizing `identity:get_user(user_id=3d92582157634b598abe103256a001ef)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:08.395118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e789ff-bab8-4686-b3a4-a6af61d4d09f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:08.397163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 152/309] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:08 2022] GET /identity/v3/users/3d92582157634b598abe103256a001ef => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:08.403758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:08.513050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cec983-6fd7-4ed4-b4e3-50968c20696c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:08.548085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be85a-b0e0-42e6-847a-8557c925cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:08.548765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be85a-b0e0-42e6-847a-8557c925cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:08.549074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be85a-b0e0-42e6-847a-8557c925cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:08.549349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be85a-b0e0-42e6-847a-8557c925cdc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:08.550198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be85a-b0e0-42e6-847a-8557c925cdc6 None admin] RBAC: Adding query filter params (role.id=070af100905e4b06b408f229d26a3bdb, user.id=3d92582157634b598abe103256a001ef, scope.system=all) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:08.550503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be85a-b0e0-42e6-847a-8557c925cdc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:08.552069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be85a-b0e0-42e6-847a-8557c925cdc6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:08.559163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 158/310] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 4 14:38:08 2022] GET /identity/v3/role_assignments?user.id=3d92582157634b598abe103256a001ef&scope.system=all&role.id=070af100905e4b06b408f229d26a3bdb => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:09.221802 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75c63f-fa3d-48a1-9e12-0173337cfff5 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:09.222359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75c63f-fa3d-48a1-9e12-0173337cfff5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:09.222359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75c63f-fa3d-48a1-9e12-0173337cfff5 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:09.224010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 153/311] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:09.229713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0a701-aad4-4857-9906-59377e77981f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:09.229998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0a701-aad4-4857-9906-59377e77981f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:09.230274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0a701-aad4-4857-9906-59377e77981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:09.247531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d5a0a701-aad4-4857-9906-59377e77981f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:09.249107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d5a0a701-aad4-4857-9906-59377e77981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:09.320535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 159/312] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:09.343443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75c63f-fa3d-48a1-9e12-0173337cfff5 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:09.446046 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75c63f-fa3d-48a1-9e12-0173337cfff5 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:09.480004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:09.480725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:09.481032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:09.481303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:09.492091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:09.499783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:09.500696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:09.501568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 154/313] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:09.509962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0a701-aad4-4857-9906-59377e77981f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:09.619854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0a701-aad4-4857-9906-59377e77981f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:09.655400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:09.656547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:09.657072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:09.657558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:09.658587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:09.659092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:09.660695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:09.670056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 160/314] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:09.676118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:09.778532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3f753-8994-46b9-8df5-2577f7eef702 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:09.812181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:09.813093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:09.813371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:09.813641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] PATH_INFO: `/v3/users/d6b08fd00cca4866907fce4c3517de7f` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:09.814854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] RBAC: Authorizing `identity:get_user(user_id=d6b08fd00cca4866907fce4c3517de7f)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:09.816341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:09.822904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 155/315] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:09 2022] GET /identity/v3/users/d6b08fd00cca4866907fce4c3517de7f => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:09.825998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:09.930885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453042d-1ba1-49d8-9030-f4aa23428ba4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:09.965862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:09.966920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:09.967426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:09.967897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.037602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:10.039425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:10.142107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 161/316] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:10.148402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:10.253719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da39b8-2176-4619-91c2-031f56281d0a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:10.291515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9604df8-a9a5-47f6-92c7-5694a6105db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:10.292319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604df8-a9a5-47f6-92c7-5694a6105db8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.292606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604df8-a9a5-47f6-92c7-5694a6105db8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.292900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604df8-a9a5-47f6-92c7-5694a6105db8 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.294111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9604df8-a9a5-47f6-92c7-5694a6105db8 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:10.295568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9604df8-a9a5-47f6-92c7-5694a6105db8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:10.302295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 156/317] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:10 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:10.304481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:10.412581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e646a-66ff-4a8c-8238-094407448834 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:10.438571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074413ba-fc4c-4d71-8a4f-2d244f9f1c52 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.438853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074413ba-fc4c-4d71-8a4f-2d244f9f1c52 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.439131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074413ba-fc4c-4d71-8a4f-2d244f9f1c52 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.440829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 157/318] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:10.446040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937977d-0a25-4b20-a036-8098d41ee942 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.446326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937977d-0a25-4b20-a036-8098d41ee942 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.446602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937977d-0a25-4b20-a036-8098d41ee942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.448946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:10.449616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.449894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.450164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.451001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] RBAC: Adding query filter params (role.id=070af100905e4b06b408f229d26a3bdb, scope.project.id=cefff7c16392441d88d3526f6461ccb4, user.id=d6b08fd00cca4866907fce4c3517de7f) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:10.451308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:10.452972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:10.464206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 162/319] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:10 2022] GET /identity/v3/role_assignments?user.id=d6b08fd00cca4866907fce4c3517de7f&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=070af100905e4b06b408f229d26a3bdb => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:10.465875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e937977d-0a25-4b20-a036-8098d41ee942 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:10.467353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e937977d-0a25-4b20-a036-8098d41ee942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:10.544603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 158/320] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:10.568340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:10.698959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e997b6a-8ae1-4165-9ab7-3bba916fe913 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:10.736724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213f9cf-47d1-46c3-ab04-283a47d8e528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:10.737975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f9cf-47d1-46c3-ab04-283a47d8e528 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.738515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f9cf-47d1-46c3-ab04-283a47d8e528 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.738791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f9cf-47d1-46c3-ab04-283a47d8e528 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.750084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213f9cf-47d1-46c3-ab04-283a47d8e528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-3213f9cf-47d1-46c3-ab04-283a47d8e528 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:10.757990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:10.758989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:10.760403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 163/321] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:10.765551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937977d-0a25-4b20-a036-8098d41ee942 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:10.801294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199b9f2-e80f-4c88-8d40-3e8606f0ba2b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.801487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199b9f2-e80f-4c88-8d40-3e8606f0ba2b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.801718 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199b9f2-e80f-4c88-8d40-3e8606f0ba2b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.803111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 164/322] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:10.809677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189fbc6-ef34-45a0-9f27-04d21c8a574d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.809958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189fbc6-ef34-45a0-9f27-04d21c8a574d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.810728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189fbc6-ef34-45a0-9f27-04d21c8a574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.827480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1189fbc6-ef34-45a0-9f27-04d21c8a574d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:10.829095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1189fbc6-ef34-45a0-9f27-04d21c8a574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:10.869977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937977d-0a25-4b20-a036-8098d41ee942 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:10.900867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 165/323] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:10.902316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b937cd-c178-42d6-9ee8-839520f352bf None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.902632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b937cd-c178-42d6-9ee8-839520f352bf None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.902863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b937cd-c178-42d6-9ee8-839520f352bf None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.903336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:10.904019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.904134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 166/324] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:10.904305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.904575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.905373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:10.905681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:10.907060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:10.910304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba46ca-5300-422d-8901-d6e5c6caefe7 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:10.910586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba46ca-5300-422d-8901-d6e5c6caefe7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:10.910861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba46ca-5300-422d-8901-d6e5c6caefe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:10.921097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 159/325] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:10.922786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:10.928258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-cdba46ca-5300-422d-8901-d6e5c6caefe7 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:10.929479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cdba46ca-5300-422d-8901-d6e5c6caefe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:10.998608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 167/326] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:10.999245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba46ca-5300-422d-8901-d6e5c6caefe7 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:11.029186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcb912-c35c-48a8-94e1-66799b2812d1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.061480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.062290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.062569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.062836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] PATH_INFO: `/v3/users/5056e3c0d51e41bbafd3e6a813c0719e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.064062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] RBAC: Authorizing `identity:get_user(user_id=5056e3c0d51e41bbafd3e6a813c0719e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:11.065549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:11.067193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 160/327] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:10 2022] GET /identity/v3/users/5056e3c0d51e41bbafd3e6a813c0719e => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:11.067962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:11.096496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba46ca-5300-422d-8901-d6e5c6caefe7 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.131234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.132080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.132378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.132638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.133864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:11.135396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:11.137114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 168/328] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:11.137884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:11.172771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b3fa7-e6f5-4617-b777-c5936e6e3386 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.206233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.207128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.207467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.207772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.218402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:11.227657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:11.228843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 161/329] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:11.229856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:11.241328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431e48a-9981-488d-b438-bf930ffda857 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.275519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.276219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.276505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.276780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.277649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] RBAC: Adding query filter params (role.id=4e8a255633994167b69cee0662997d5e, user.id=5056e3c0d51e41bbafd3e6a813c0719e, scope.system=all) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:11.277960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:11.279409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:11.291715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 169/330] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 4 14:38:11 2022] GET /identity/v3/role_assignments?user.id=5056e3c0d51e41bbafd3e6a813c0719e&scope.system=all&role.id=4e8a255633994167b69cee0662997d5e => generated 510 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:11.292934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:11.348759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89bd38-2d73-4263-b09c-47778a839f1c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.386327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.387023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.387308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.387586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] PATH_INFO: `/v3/groups` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.388320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:11.389844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:11.408251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b4a2f-0e19-4294-a877-6328731cb77f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.419540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 162/331] 10.222.0.32 () {60 vars in 1134 bytes} [Wed May 4 14:38:11 2022] POST /identity/v3/groups => generated 226 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:11.449852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.450566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.450854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.451129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.451960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:11.452269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:11.453939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:11.464511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 170/332] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:11.470840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:11.594788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa043cf-fb9b-4a36-9cf1-aeb10936f28a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.631480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94b955-c66d-4ec3-9c01-49e9551585dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.632398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94b955-c66d-4ec3-9c01-49e9551585dc None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.632679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94b955-c66d-4ec3-9c01-49e9551585dc None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.632984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94b955-c66d-4ec3-9c01-49e9551585dc None admin] PATH_INFO: `/v3/users/3d92582157634b598abe103256a001ef` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.634193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94b955-c66d-4ec3-9c01-49e9551585dc None admin] RBAC: Authorizing `identity:get_user(user_id=3d92582157634b598abe103256a001ef)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:11.635730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94b955-c66d-4ec3-9c01-49e9551585dc None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:11.637427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 163/333] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:11 2022] GET /identity/v3/users/3d92582157634b598abe103256a001ef => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:11.643305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:11.748539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f690c-7071-4a3e-a0b4-a0fb337cab57 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:11.783664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021eebf5-84ac-40f9-87c8-8c4787780012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:11.784537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eebf5-84ac-40f9-87c8-8c4787780012 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:11.784855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eebf5-84ac-40f9-87c8-8c4787780012 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:11.785127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021eebf5-84ac-40f9-87c8-8c4787780012 None admin] PATH_INFO: `/v3/system/users/3d92582157634b598abe103256a001ef/roles/070af100905e4b06b408f229d26a3bdb` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:11.786636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021eebf5-84ac-40f9-87c8-8c4787780012 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3d92582157634b598abe103256a001ef, role_id=070af100905e4b06b408f229d26a3bdb)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:11.788034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021eebf5-84ac-40f9-87c8-8c4787780012 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:11.796814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 171/334] 10.222.0.32 () {58 vars in 1334 bytes} [Wed May 4 14:38:11 2022] PUT /identity/v3/system/users/3d92582157634b598abe103256a001ef/roles/070af100905e4b06b408f229d26a3bdb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:13.915426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54dcbb-7b64-4048-8800-4a5173b540ae None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:13.917851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54dcbb-7b64-4048-8800-4a5173b540ae None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:13.918141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54dcbb-7b64-4048-8800-4a5173b540ae None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:13.919938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 164/335] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:13.926664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011db9-13a3-4aa0-b0a2-c4988d0224e6 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:13.927141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011db9-13a3-4aa0-b0a2-c4988d0224e6 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:13.927214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011db9-13a3-4aa0-b0a2-c4988d0224e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:13.949692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6a011db9-13a3-4aa0-b0a2-c4988d0224e6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:13.950132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6a011db9-13a3-4aa0-b0a2-c4988d0224e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:14.035766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 172/336] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:14.075058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54dcbb-7b64-4048-8800-4a5173b540ae None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:14.080112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e765c8-c0d4-4bde-ad72-88c9ac81b0ee None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:14.080389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e765c8-c0d4-4bde-ad72-88c9ac81b0ee None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:14.080652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e765c8-c0d4-4bde-ad72-88c9ac81b0ee None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:14.082343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 173/337] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:14.090300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae7484-104e-4e9c-9649-e182b3b64617 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:14.090575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae7484-104e-4e9c-9649-e182b3b64617 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:14.090839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae7484-104e-4e9c-9649-e182b3b64617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:14.110633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-39ae7484-104e-4e9c-9649-e182b3b64617 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:14.111822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-39ae7484-104e-4e9c-9649-e182b3b64617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:14.201047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 174/338] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:14.207998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54dcbb-7b64-4048-8800-4a5173b540ae None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:14.249339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae7484-104e-4e9c-9649-e182b3b64617 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:14.250177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:14.251048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:14.251340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:14.251616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:14.270595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:14.278906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:14.281043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:14.282999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:14.286371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 165/339] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:14.291200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:14.398403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae7484-104e-4e9c-9649-e182b3b64617 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:14.436256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa618d19-6316-455b-8294-0526dc44869c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:14.448337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:14.449205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:14.449481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:14.449746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:14.469979 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:14.478101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:14.479113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:14.479925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 175/340] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:14.485466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:14.488145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:14.488855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:14.489144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:14.489420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:14.490192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:14.490502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:14.491887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:14.505597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 166/341] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:14.511456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:14.638654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed406700-ef97-47ad-92f6-050f3dc06768 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:14.685110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:14.685846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:14.686137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:14.686419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:14.703622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b054d3-b713-4ff3-b347-e4f1f910abf0 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:14.768420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:14.769238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:14.769515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:14.769785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:14.770569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:14.770873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:14.772332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:14.794489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 176/342] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:14.800397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:14.835980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:14.837519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:14.960016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 167/343] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:14.965645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:14.968459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc97e8-be21-484f-9702-e2590bdac3d5 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.005664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02ca8b-b9a9-4b65-a163-9f6e5a84a4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.006730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02ca8b-b9a9-4b65-a163-9f6e5a84a4aa None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.007219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02ca8b-b9a9-4b65-a163-9f6e5a84a4aa None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.007677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02ca8b-b9a9-4b65-a163-9f6e5a84a4aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.077167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02ca8b-b9a9-4b65-a163-9f6e5a84a4aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:15.079084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02ca8b-b9a9-4b65-a163-9f6e5a84a4aa None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:15.137959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a435a-06f8-4fff-92e7-b21954328a60 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.181275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 177/344] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:15.183067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e87d2a-7a5e-466a-88d8-5c7322a6b511 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.183711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e87d2a-7a5e-466a-88d8-5c7322a6b511 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.184186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e87d2a-7a5e-466a-88d8-5c7322a6b511 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.186097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 178/345] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:15.187066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e87d2a-7a5e-466a-88d8-5c7322a6b511 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.198935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3382f-4f58-4ae5-99ed-0bfdf996c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.199839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3382f-4f58-4ae5-99ed-0bfdf996c957 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.200118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3382f-4f58-4ae5-99ed-0bfdf996c957 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.200393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3382f-4f58-4ae5-99ed-0bfdf996c957 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.201723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3382f-4f58-4ae5-99ed-0bfdf996c957 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:15.203206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3382f-4f58-4ae5-99ed-0bfdf996c957 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:15.205355 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 168/346] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:14 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:15.206588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae06a1b-9178-43e3-ad25-580f89da68d9 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.206879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae06a1b-9178-43e3-ad25-580f89da68d9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.207134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae06a1b-9178-43e3-ad25-580f89da68d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.233824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6ae06a1b-9178-43e3-ad25-580f89da68d9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:15.235301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6ae06a1b-9178-43e3-ad25-580f89da68d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:15.298401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e87d2a-7a5e-466a-88d8-5c7322a6b511 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.334789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.336036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.336604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.337092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.338617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:15.340341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:15.342450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 179/347] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:15.343439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.363207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 169/348] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:15.364085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae06a1b-9178-43e3-ad25-580f89da68d9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.468229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411a69c-ffa1-4d0f-95d2-2720c3eaaa02 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.497676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae06a1b-9178-43e3-ad25-580f89da68d9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.506265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75320d48-6a39-426c-af64-e93a3b90acc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.507362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75320d48-6a39-426c-af64-e93a3b90acc1 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.507817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75320d48-6a39-426c-af64-e93a3b90acc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.508254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75320d48-6a39-426c-af64-e93a3b90acc1 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.514875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75320d48-6a39-426c-af64-e93a3b90acc1 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:15.516593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75320d48-6a39-426c-af64-e93a3b90acc1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:15.519101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 180/349] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:15.520518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d775216-a5ef-4733-9468-7b3b2526d2dd None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.520997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d775216-a5ef-4733-9468-7b3b2526d2dd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.521440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d775216-a5ef-4733-9468-7b3b2526d2dd None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.523358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 181/350] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:15.524256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d775216-a5ef-4733-9468-7b3b2526d2dd None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.538682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0668-3a9d-4df0-a549-67154fed7d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.539953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0668-3a9d-4df0-a549-67154fed7d50 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.540303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0668-3a9d-4df0-a549-67154fed7d50 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.540641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0668-3a9d-4df0-a549-67154fed7d50 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.542206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0668-3a9d-4df0-a549-67154fed7d50 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:15.544250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0668-3a9d-4df0-a549-67154fed7d50 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:15.546592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 170/351] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:15.548251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476bd5-f4fa-4503-b4db-bd5d7b5d4e1a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.548594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476bd5-f4fa-4503-b4db-bd5d7b5d4e1a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.548975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476bd5-f4fa-4503-b4db-bd5d7b5d4e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.571216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-72476bd5-f4fa-4503-b4db-bd5d7b5d4e1a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:15.573088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-72476bd5-f4fa-4503-b4db-bd5d7b5d4e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:15.635692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d775216-a5ef-4733-9468-7b3b2526d2dd None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.640775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 171/352] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:15.641790 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476bd5-f4fa-4503-b4db-bd5d7b5d4e1a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.671582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.672818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.673333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.673795 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.686845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:15.695249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:15.696294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 04 14:38:15.697215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:15.699486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 182/353] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:15.700593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.751853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476bd5-f4fa-4503-b4db-bd5d7b5d4e1a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.788843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.789537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.789805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.790075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.790973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=c7555237bf4842dd8f28617418adec46) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:15.791286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:15.792767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:15.798629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 172/354] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=4e8a255633994167b69cee0662997d5e => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:15.799444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.808445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba628f-ed52-4d81-9e82-48f24fb59557 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.846841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:15.847952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:15.848436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:15.848961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:15.850101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=c7555237bf4842dd8f28617418adec46) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:15.850656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:15.852458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:15.859292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 183/355] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:15.860460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:15.968943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992d5f2-3469-428e-bde1-1b75e021c806 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:15.981039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee197390-4801-4dd2-853f-da241bd48adb None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:16.009250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8acaf-8b1c-4d13-94fe-d68eae19b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.010393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8acaf-8b1c-4d13-94fe-d68eae19b489 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.010677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8acaf-8b1c-4d13-94fe-d68eae19b489 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.010977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8acaf-8b1c-4d13-94fe-d68eae19b489 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.028824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8acaf-8b1c-4d13-94fe-d68eae19b489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-58c8acaf-8b1c-4d13-94fe-d68eae19b489 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:16.034613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:16.035566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:16.036375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 173/356] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:16.037665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4747e6-446a-49bc-9c41-0f85e694f76d None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.037665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4747e6-446a-49bc-9c41-0f85e694f76d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.037829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4747e6-446a-49bc-9c41-0f85e694f76d None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.039156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 174/357] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:16.044381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4747e6-446a-49bc-9c41-0f85e694f76d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.046754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64cc88-6a3b-49a4-b847-fbbf8d35644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.047808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64cc88-6a3b-49a4-b847-fbbf8d35644f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.048302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64cc88-6a3b-49a4-b847-fbbf8d35644f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.048870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64cc88-6a3b-49a4-b847-fbbf8d35644f None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.049902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64cc88-6a3b-49a4-b847-fbbf8d35644f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:16.050416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64cc88-6a3b-49a4-b847-fbbf8d35644f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:16.052155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd64cc88-6a3b-49a4-b847-fbbf8d35644f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:16.070550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 184/358] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:16.072414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcabb1-22a2-4206-be1b-e38d0ce787e8 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.073014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcabb1-22a2-4206-be1b-e38d0ce787e8 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.073493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcabb1-22a2-4206-be1b-e38d0ce787e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.097267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b8bcabb1-22a2-4206-be1b-e38d0ce787e8 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:16.098841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b8bcabb1-22a2-4206-be1b-e38d0ce787e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:16.159657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4747e6-446a-49bc-9c41-0f85e694f76d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:16.184364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 185/359] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:16.185310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcabb1-22a2-4206-be1b-e38d0ce787e8 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.228928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.229539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.230514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.230514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.230826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:16.231140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:16.232515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:16.248214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 175/360] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:16.249111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.313356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcabb1-22a2-4206-be1b-e38d0ce787e8 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:16.352873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10863d6e-af0e-44bb-b088-02511b18698b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:16.354735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.355853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.356329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.356850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] PATH_INFO: `/v3/users/3d92582157634b598abe103256a001ef` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.358288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] RBAC: Authorizing `identity:get_user(user_id=3d92582157634b598abe103256a001ef)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:16.359974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:16.361919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 186/361] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/users/3d92582157634b598abe103256a001ef => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:16.362905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.393196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.394060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.394348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.394618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.406405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:16.414728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:16.415689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:16.416557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 176/362] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:16.421489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.473545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b894-e025-4bd9-86d9-f980d2001d45 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:16.510569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19538e08-7290-4898-a7a5-85265337028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.511595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19538e08-7290-4898-a7a5-85265337028f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.512070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19538e08-7290-4898-a7a5-85265337028f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.512538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19538e08-7290-4898-a7a5-85265337028f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.531204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39712d-4d5f-45af-8311-3f8f93d2fbb5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:16.579237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.580293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.580293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.580482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.581310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] RBAC: Adding query filter params (role.id=070af100905e4b06b408f229d26a3bdb, user.id=3d92582157634b598abe103256a001ef, scope.system=all) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:16.581622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:16.583068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:16.583398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19538e08-7290-4898-a7a5-85265337028f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:16.585134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19538e08-7290-4898-a7a5-85265337028f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:16.600514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 177/363] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/role_assignments?user.id=3d92582157634b598abe103256a001ef&scope.system=all&role.id=070af100905e4b06b408f229d26a3bdb => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:16.601403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.690953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 187/364] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:16.696964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19538e08-7290-4898-a7a5-85265337028f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.816058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1d97b-70b4-4996-896a-4b6c4568dd00 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:16.878912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:16.880009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:16.880494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:16.881020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:16.882077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:16.882611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:16.885536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:16.898276 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 178/365] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:16.904643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:16.951090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19538e08-7290-4898-a7a5-85265337028f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.019208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9fae5-6c6c-45f7-b6c9-ef4151332e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.020095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9fae5-6c6c-45f7-b6c9-ef4151332e2e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.020375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9fae5-6c6c-45f7-b6c9-ef4151332e2e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.020652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9fae5-6c6c-45f7-b6c9-ef4151332e2e None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.021952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9fae5-6c6c-45f7-b6c9-ef4151332e2e None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.023802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9fae5-6c6c-45f7-b6c9-ef4151332e2e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.030775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 188/366] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:17.032274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8871c-974f-4c69-9f32-720eb0d478b9 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.032567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8871c-974f-4c69-9f32-720eb0d478b9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.032879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8871c-974f-4c69-9f32-720eb0d478b9 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.034512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 189/367] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:17.035234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8871c-974f-4c69-9f32-720eb0d478b9 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:17.055479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4800dc5-2b86-46d1-8ee2-fd2d6d94aa7e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.107191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.108614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.109081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.109772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.163456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8871c-974f-4c69-9f32-720eb0d478b9 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.202982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.205027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.205930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939da2d1-f5eb-45dc-a4b2-aad7eed20f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.206891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da2d1-f5eb-45dc-a4b2-aad7eed20f15 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.207247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da2d1-f5eb-45dc-a4b2-aad7eed20f15 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.207704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da2d1-f5eb-45dc-a4b2-aad7eed20f15 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.209542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939da2d1-f5eb-45dc-a4b2-aad7eed20f15 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.210679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939da2d1-f5eb-45dc-a4b2-aad7eed20f15 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.212499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 190/368] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:17.214056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b030851-e2d1-4f0f-b4cf-fbe1c8ee69fa None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.214412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b030851-e2d1-4f0f-b4cf-fbe1c8ee69fa None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.214761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b030851-e2d1-4f0f-b4cf-fbe1c8ee69fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.235580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8b030851-e2d1-4f0f-b4cf-fbe1c8ee69fa None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:17.237012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8b030851-e2d1-4f0f-b4cf-fbe1c8ee69fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:17.314582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 191/369] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:17.315552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b030851-e2d1-4f0f-b4cf-fbe1c8ee69fa None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:17.331000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 179/370] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:17.336517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:17.449943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b030851-e2d1-4f0f-b4cf-fbe1c8ee69fa None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.480159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd09a-fb5b-4769-83ee-cae59f0dc6aa None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.490680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.491368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.492048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.492353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.493569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=cefff7c16392441d88d3526f6461ccb4) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:17.493895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.495914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.504596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 192/371] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=4e8a255633994167b69cee0662997d5e => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:17.505919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:17.520777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.521748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.522027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.522297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.523979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.525595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.527819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 180/372] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:17.533616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:17.629260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7930e-b978-409d-aa6a-babd4984f324 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.667788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.668627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.668951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.669127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c18f4-d5df-41e9-a13a-0f5118405015 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.669223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.683709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:17.693717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:17.695947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:17.697990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 193/373] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:17.704918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:17.723751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.724675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.724979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.725261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.728146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.729788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.732680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 181/374] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:17.738533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:17.879617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253a668-ee11-4c13-9c47-28de28f095a5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.890899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c257f6c-a340-4ce0-94ea-7afacaf3a13c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:17.927551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.928263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.928554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.928850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.929710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=cefff7c16392441d88d3526f6461ccb4) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:17.930030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.931766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.945381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 182/375] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:17.961709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:17.962340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:17.962620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:17.963019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:17.963847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:17.964255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:17.965718 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:17.975797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 194/376] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:17.981748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:18.119496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31661a0f-858b-4450-8134-a3249da76a55 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:18.167723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:18.168407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:18.168691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:18.168991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:18.246913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:18.248347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:18.352782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 183/377] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:18.358198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:18.504006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0d7de-8a3f-4faf-a557-12a8d8405434 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:18.545238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:18.546205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:18.546487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:18.546763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] PATH_INFO: `/v3/projects/62c598d6329d4180a1bbd38c5500b8cc` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:18.548059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] RBAC: Authorizing `identity:get_project(project_id=62c598d6329d4180a1bbd38c5500b8cc)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:18.549584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:18.551720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 195/378] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:18 2022] GET /identity/v3/projects/62c598d6329d4180a1bbd38c5500b8cc => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:18.562016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:18.697992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef09995-e941-4b2d-a077-bcd4094b710e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:18.740252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9302d7c-54e3-4ce7-bfe7-34d431afd79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:18.741167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9302d7c-54e3-4ce7-bfe7-34d431afd79a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:18.741446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9302d7c-54e3-4ce7-bfe7-34d431afd79a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:18.741719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9302d7c-54e3-4ce7-bfe7-34d431afd79a None admin] PATH_INFO: `/v3/groups/12f4d7fe42c84263949f3c0406ba1d76` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:18.753477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9302d7c-54e3-4ce7-bfe7-34d431afd79a None admin] RBAC: Authorizing `identity:get_group(group_id=12f4d7fe42c84263949f3c0406ba1d76)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:18.755030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9302d7c-54e3-4ce7-bfe7-34d431afd79a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:18.756749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 184/379] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:18 2022] GET /identity/v3/groups/12f4d7fe42c84263949f3c0406ba1d76 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:18.762951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:18.887377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ab00-673f-4c1f-b991-8037550b504d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:18.929497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca0757-b4a8-4253-aff6-bf25d288bf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:18.930530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0757-b4a8-4253-aff6-bf25d288bf64 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:18.931000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0757-b4a8-4253-aff6-bf25d288bf64 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:18.931552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca0757-b4a8-4253-aff6-bf25d288bf64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:18.932623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca0757-b4a8-4253-aff6-bf25d288bf64 None admin] RBAC: Adding query filter params (group.id=12f4d7fe42c84263949f3c0406ba1d76, role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=62c598d6329d4180a1bbd38c5500b8cc) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:18.933172 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca0757-b4a8-4253-aff6-bf25d288bf64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:18.934797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca0757-b4a8-4253-aff6-bf25d288bf64 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:18.947201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 196/380] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:18 2022] GET /identity/v3/role_assignments?group.id=12f4d7fe42c84263949f3c0406ba1d76&scope.project.id=62c598d6329d4180a1bbd38c5500b8cc&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:19.136967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188ea16-1742-4c66-8273-88ed3c727eff None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:19.137465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188ea16-1742-4c66-8273-88ed3c727eff None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:19.137465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188ea16-1742-4c66-8273-88ed3c727eff None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:19.139189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 185/381] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:19.145166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0720d-8061-4b93-8c0b-99bf39fd7462 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:19.145834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0720d-8061-4b93-8c0b-99bf39fd7462 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:19.146303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0720d-8061-4b93-8c0b-99bf39fd7462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:19.170989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-41a0720d-8061-4b93-8c0b-99bf39fd7462 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:19.172611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-41a0720d-8061-4b93-8c0b-99bf39fd7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:19.257315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 197/382] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:19.283586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188ea16-1742-4c66-8273-88ed3c727eff None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:19.405006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188ea16-1742-4c66-8273-88ed3c727eff None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:19.417625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847ccb8-f7a4-4fb4-bc9b-2fdd13097896 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:19.418806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847ccb8-f7a4-4fb4-bc9b-2fdd13097896 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:19.419292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847ccb8-f7a4-4fb4-bc9b-2fdd13097896 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:19.421842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 198/383] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:19.428045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0163c-5c4b-46e0-b415-2aebd8c23308 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:19.429083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0163c-5c4b-46e0-b415-2aebd8c23308 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:19.429565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0163c-5c4b-46e0-b415-2aebd8c23308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:19.450088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a2a0163c-5c4b-46e0-b415-2aebd8c23308 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:19.451959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a2a0163c-5c4b-46e0-b415-2aebd8c23308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:19.458103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:19.458937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:19.459216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:19.459492 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:19.485685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:19.498126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:19.499087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:19.499863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:19.500466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 186/384] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:19.513902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:19.578509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 199/385] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:19.611821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0163c-5c4b-46e0-b415-2aebd8c23308 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:19.694289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb7e91-13b7-43cd-8bba-003ae5df7bb6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:19.736308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:19.737043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:19.737331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:19.737609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:19.738401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:19.738711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:19.740092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:19.750525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 187/386] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:19.757842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:19.768089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0163c-5c4b-46e0-b415-2aebd8c23308 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:19.822443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:19.823285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:19.823604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:19.823832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:19.845683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:19.853830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:19.854819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:19.855624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 200/387] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:19.862543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:19.962889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdd83d-1d40-4f6d-b9bf-d48d6c80be8f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:19.976052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6db5-5d93-4cda-8e31-86e0da45068b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:20.010212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.010902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.011203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.011454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.012214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:20.012521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:20.013975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:20.023041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 201/388] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:20.030085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:20.032585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.033721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.034214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.034691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.041634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:20.043412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:20.045689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 188/389] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:19 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:20.051406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:20.136347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021e2e5-04db-4e3b-b53c-865c2abee3e0 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:20.170658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.171551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.171830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.172103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.173394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:20.174896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:20.176600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 202/390] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:20 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:20.182222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:20.231682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521957-dc24-4635-9f03-e592b3fb80d3 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:20.289439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ee737-7852-46ee-ac35-a2e55b4d6b55 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:20.321830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.322696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.322981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.323277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.325225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.326301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.326751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.327210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.328636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:20.329837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:20.330532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:20.331400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:20.332997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 203/391] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:20 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:20.336920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 189/392] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:20 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:20.339259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:20.339816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:20.439487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673b718-8eea-44ee-a924-8969c2b69fe6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:20.474353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545ca13-40d9-4d75-b336-8b34c824cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.475555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545ca13-40d9-4d75-b336-8b34c824cb16 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.476032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545ca13-40d9-4d75-b336-8b34c824cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.476474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545ca13-40d9-4d75-b336-8b34c824cb16 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46/groups/353d3434c82e41859ed00edffbbf04d1/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.479117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545ca13-40d9-4d75-b336-8b34c824cb16 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7555237bf4842dd8f28617418adec46, group_id=353d3434c82e41859ed00edffbbf04d1, role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:20.480786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545ca13-40d9-4d75-b336-8b34c824cb16 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:20.504332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 204/393] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 4 14:38:20 2022] PUT /identity/v3/projects/c7555237bf4842dd8f28617418adec46/groups/353d3434c82e41859ed00edffbbf04d1/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:20.524563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca85131-0b80-46cc-9951-7ed0742ad451 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:20.574014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.574956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.575239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.575573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46/groups/353d3434c82e41859ed00edffbbf04d1/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.580945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7555237bf4842dd8f28617418adec46, group_id=353d3434c82e41859ed00edffbbf04d1, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:20.582462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:20.593131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561a7d8-50fa-4810-87c0-94e6a1913f55 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.593747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561a7d8-50fa-4810-87c0-94e6a1913f55 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.594241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561a7d8-50fa-4810-87c0-94e6a1913f55 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.596332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 205/394] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:20.600549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 190/395] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 4 14:38:20 2022] PUT /identity/v3/projects/c7555237bf4842dd8f28617418adec46/groups/353d3434c82e41859ed00edffbbf04d1/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:38:20.605762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f823-1d09-4cf9-8a65-66596b4f9476 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.606343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f823-1d09-4cf9-8a65-66596b4f9476 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.606831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131f823-1d09-4cf9-8a65-66596b4f9476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.629722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1131f823-1d09-4cf9-8a65-66596b4f9476 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:20.631264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1131f823-1d09-4cf9-8a65-66596b4f9476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:20.712047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 206/396] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:20.739067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:20.861481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcabd3-83a5-41b9-9eeb-15751c710432 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:20.909234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:20.910086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:20.910365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:20.910636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:20.925277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:20.933880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:20.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:20.935634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:20.938594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 191/397] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:20.944727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f823-1d09-4cf9-8a65-66596b4f9476 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:21.067295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131f823-1d09-4cf9-8a65-66596b4f9476 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:21.106879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e7620-521c-4849-aed0-f86e2d28eaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:21.107574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e7620-521c-4849-aed0-f86e2d28eaaf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.107856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e7620-521c-4849-aed0-f86e2d28eaaf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.108132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e7620-521c-4849-aed0-f86e2d28eaaf None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.108949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e7620-521c-4849-aed0-f86e2d28eaaf None admin] RBAC: Adding query filter params (name=member) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:21.109260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e7620-521c-4849-aed0-f86e2d28eaaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:21.110806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e7620-521c-4849-aed0-f86e2d28eaaf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:21.125203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 207/398] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:21.131084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:21.263791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc3efe-4abe-46a6-abe3-0af1118ce669 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:21.298529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a595baf-ab1e-4041-a0c9-f8d233bfcc86 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.299196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a595baf-ab1e-4041-a0c9-f8d233bfcc86 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.299664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a595baf-ab1e-4041-a0c9-f8d233bfcc86 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.301374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:21.301899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 208/399] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:21.302224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.302506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.302780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.304030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:21.305492 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:21.307575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 192/400] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:21 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:21.309449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6a62d-d302-46ac-899f-f2c6582f434d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.310070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6a62d-d302-46ac-899f-f2c6582f434d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.310559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6a62d-d302-46ac-899f-f2c6582f434d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.321171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:21.350331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-27a6a62d-d302-46ac-899f-f2c6582f434d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:21.351540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-27a6a62d-d302-46ac-899f-f2c6582f434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:21.465056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 209/401] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:21.489440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6a62d-d302-46ac-899f-f2c6582f434d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:21.565361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e0647-ae67-4ade-a7cf-ea287b6c548b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:21.609311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6a62d-d302-46ac-899f-f2c6582f434d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:21.646820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:21.648076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.648349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.648620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.653611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:21.654203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:21.655180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:21.655558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.656054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.656557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.657402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 193/402] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:21 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:21.665380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:21.669606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:21.679554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:21.680504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:21.681375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:21.689326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 210/403] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:21.692507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:21.879992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734cc6d1-2f4f-4786-b815-8ff5291fb851 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:21.909044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefba2f7-4d51-403e-85d5-00f19bc1ca14 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:21.959225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:21.959462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:21.960139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.960414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.960691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.961157 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:21.961494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:21.961726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:21.961850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:21.964699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4/groups/353d3434c82e41859ed00edffbbf04d1/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:21.964971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:21.967543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] RBAC: Authorizing `identity:create_grant(project_id=cefff7c16392441d88d3526f6461ccb4, group_id=353d3434c82e41859ed00edffbbf04d1, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:21.969226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:21.981660 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 211/404] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:21.984764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:21.996550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 194/405] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 4 14:38:21 2022] PUT /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4/groups/353d3434c82e41859ed00edffbbf04d1/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:38:22.115147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41e7d9-ce8f-46bb-ae5a-2047dde30eac None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:22.152489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8280ed-79ab-4318-a5c5-1bb358ad94c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:22.153679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8280ed-79ab-4318-a5c5-1bb358ad94c8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:22.154104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8280ed-79ab-4318-a5c5-1bb358ad94c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:22.154517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8280ed-79ab-4318-a5c5-1bb358ad94c8 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:22.157852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8280ed-79ab-4318-a5c5-1bb358ad94c8 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:22.159658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8280ed-79ab-4318-a5c5-1bb358ad94c8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:22.166899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 212/406] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:21 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:22.168598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:22.219564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f697f-25b5-460c-96e3-fb04bb5fac26 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:22.220088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f697f-25b5-460c-96e3-fb04bb5fac26 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:22.220506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f697f-25b5-460c-96e3-fb04bb5fac26 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:22.222174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 213/407] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:22.228015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713589-36d1-449f-a450-390655df4b5f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:22.228341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713589-36d1-449f-a450-390655df4b5f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:22.228626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40713589-36d1-449f-a450-390655df4b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:22.258888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-40713589-36d1-449f-a450-390655df4b5f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:22.259426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-40713589-36d1-449f-a450-390655df4b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:22.290832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059436d1-2596-44d3-b530-b7b927597cca None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:22.340468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:22.342465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:22.342743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:22.343021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:22.348020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:22.350509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:22.352401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 195/408] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:22 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:22.363794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:22.392525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 214/409] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:22.432002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40713589-36d1-449f-a450-390655df4b5f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:22.530980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc482e-60d6-43e1-93d1-da74a2c21f96 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:22.555879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40713589-36d1-449f-a450-390655df4b5f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:22.593659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:22.594606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:22.594889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:22.595164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:22.601565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:22.602664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:22.602952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:22.603295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4/groups/353d3434c82e41859ed00edffbbf04d1/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:22.605643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] RBAC: Authorizing `identity:create_grant(project_id=cefff7c16392441d88d3526f6461ccb4, group_id=353d3434c82e41859ed00edffbbf04d1, role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:22.607104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:22.607761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:22.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:22.620640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:22.621684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:22.624499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 215/410] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:22.632881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:22.634167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 196/411] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 4 14:38:22 2022] PUT /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4/groups/353d3434c82e41859ed00edffbbf04d1/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:22.768544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f08454-dec1-4f93-b5c9-ba375055dbd1 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:22.811563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:22.812251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:22.812574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:22.812874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:22.813675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:22.813987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:22.815462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:22.827113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 216/412] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:22.832991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:22.965633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8425548-971a-4e02-8a18-936f33464105 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:23.006782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:23.007661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:23.007940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:23.008227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] PATH_INFO: `/v3/groups/12f4d7fe42c84263949f3c0406ba1d76` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:23.009479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] RBAC: Authorizing `identity:get_group(group_id=12f4d7fe42c84263949f3c0406ba1d76)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:23.010928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:23.012829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 197/413] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:22 2022] GET /identity/v3/groups/12f4d7fe42c84263949f3c0406ba1d76 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:23.018383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:23.139424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe9c01-5a5b-4cc5-a9a0-79b70aa7a877 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:23.175967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec63fe-e8c1-447b-90d8-1ae4d91c80a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:23.177362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec63fe-e8c1-447b-90d8-1ae4d91c80a0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:23.177842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec63fe-e8c1-447b-90d8-1ae4d91c80a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:23.178341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec63fe-e8c1-447b-90d8-1ae4d91c80a0 None admin] PATH_INFO: `/v3/projects/62c598d6329d4180a1bbd38c5500b8cc` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:23.182471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec63fe-e8c1-447b-90d8-1ae4d91c80a0 None admin] RBAC: Authorizing `identity:get_project(project_id=62c598d6329d4180a1bbd38c5500b8cc)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:23.184261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec63fe-e8c1-447b-90d8-1ae4d91c80a0 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:23.186589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 217/414] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:23 2022] GET /identity/v3/projects/62c598d6329d4180a1bbd38c5500b8cc => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:23.191998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:23.341291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5947ef-a29b-43e6-b3e0-d7a280481bd0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:23.390158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf2fce-4c44-4aa4-a380-897a3880f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:23.390974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2fce-4c44-4aa4-a380-897a3880f189 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:23.391251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2fce-4c44-4aa4-a380-897a3880f189 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:23.391520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2fce-4c44-4aa4-a380-897a3880f189 None admin] PATH_INFO: `/v3/projects/62c598d6329d4180a1bbd38c5500b8cc/groups/12f4d7fe42c84263949f3c0406ba1d76/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:23.393519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf2fce-4c44-4aa4-a380-897a3880f189 None admin] RBAC: Authorizing `identity:create_grant(project_id=62c598d6329d4180a1bbd38c5500b8cc, group_id=12f4d7fe42c84263949f3c0406ba1d76, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:23.394955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf2fce-4c44-4aa4-a380-897a3880f189 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:23.415081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 198/415] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 4 14:38:23 2022] PUT /identity/v3/projects/62c598d6329d4180a1bbd38c5500b8cc/groups/12f4d7fe42c84263949f3c0406ba1d76/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:23.718945 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457c5d-2724-4c5a-9fa6-d39235e5f2b6 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:23.719468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457c5d-2724-4c5a-9fa6-d39235e5f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:23.719468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457c5d-2724-4c5a-9fa6-d39235e5f2b6 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:23.720750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 218/416] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:23.726731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d580e1-f012-40a1-9e39-280a3a5cfb21 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:23.727014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d580e1-f012-40a1-9e39-280a3a5cfb21 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:23.727289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d580e1-f012-40a1-9e39-280a3a5cfb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:23.744869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-16d580e1-f012-40a1-9e39-280a3a5cfb21 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:23.746077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-16d580e1-f012-40a1-9e39-280a3a5cfb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:23.814246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 199/417] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:23.826795 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09802153-92d1-4872-ae14-a4a4825b8c45 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:23.827107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09802153-92d1-4872-ae14-a4a4825b8c45 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:23.827341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09802153-92d1-4872-ae14-a4a4825b8c45 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:23.828899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 219/418] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:23.834568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09495a31-f3a2-41fa-b4bf-9f741ee81229 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:23.834858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09495a31-f3a2-41fa-b4bf-9f741ee81229 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:23.835133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09495a31-f3a2-41fa-b4bf-9f741ee81229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:23.851881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09802153-92d1-4872-ae14-a4a4825b8c45 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:23.855680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-09495a31-f3a2-41fa-b4bf-9f741ee81229 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:23.857832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-09495a31-f3a2-41fa-b4bf-9f741ee81229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:23.967655 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 200/419] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:23.991672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09802153-92d1-4872-ae14-a4a4825b8c45 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.006727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09495a31-f3a2-41fa-b4bf-9f741ee81229 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:24.025723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.026541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.026824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.027098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.037834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:24.045537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:24.046465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:24.047682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 220/420] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:24.054554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:24.163996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dff18-ee5a-4129-ab6e-98853494cef4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.198052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.198757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.199038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.199336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.200083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] RBAC: Adding query filter params (name=member) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:24.200401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:24.200548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09495a31-f3a2-41fa-b4bf-9f741ee81229 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.201823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:24.210869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 221/421] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:24.216574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:24.284334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.286181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.286974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.287824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.325612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:24.328921 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ebb1f-e6c8-4c20-989a-644c1d2c897e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:24.343903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:24.345261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:24.346073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:24.349964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 201/422] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:24.355138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:24.363680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.364342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.364622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.365711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.436826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:24.436826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:24.543178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 222/423] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:38:24.551748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:24.639202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82812-533d-4934-8253-149804dc8d84 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.666932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0db1e-06fc-447d-8fee-c27340b82341 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.713154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.713842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.714124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.714396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.714598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.715162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:24.715466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:24.715575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.715859 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.716241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.716881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:24.721659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:24.723171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:24.726228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 223/425] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:24.726356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 202/425] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:24.732170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:24.732643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:24.868407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb0f8a-ceb9-4559-a139-c472cd326572 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.922413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894ad6-2246-484c-9176-336fec1e8e0b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:24.937864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3db098-488d-461d-9806-94af537045d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.938551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db098-488d-461d-9806-94af537045d0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.938830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db098-488d-461d-9806-94af537045d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.939104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3db098-488d-461d-9806-94af537045d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.967333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00e0d9-5c7f-412f-a23b-8bafb0ecbb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:24.968203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e0d9-5c7f-412f-a23b-8bafb0ecbb09 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.968490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e0d9-5c7f-412f-a23b-8bafb0ecbb09 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.969097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00e0d9-5c7f-412f-a23b-8bafb0ecbb09 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.970744 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00e0d9-5c7f-412f-a23b-8bafb0ecbb09 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:24.973009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00e0d9-5c7f-412f-a23b-8bafb0ecbb09 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:24.975959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 224/426] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:24.978421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71827bb-b234-4832-b795-c585229cb196 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:24.978897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71827bb-b234-4832-b795-c585229cb196 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:24.979344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71827bb-b234-4832-b795-c585229cb196 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:24.982288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 225/427] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:24.983251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71827bb-b234-4832-b795-c585229cb196 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:25.016744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3db098-488d-461d-9806-94af537045d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:25.018507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3db098-488d-461d-9806-94af537045d0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:25.089006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71827bb-b234-4832-b795-c585229cb196 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:25.121320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 203/428] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:38:25.121320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1e168-7e9f-46e5-8dea-dd3ab884cd03 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.121320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1e168-7e9f-46e5-8dea-dd3ab884cd03 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.121320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1e168-7e9f-46e5-8dea-dd3ab884cd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.132726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:25.134104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.134670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.135139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.136173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=c7555237bf4842dd8f28617418adec46) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:25.136815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:25.137599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-65b1e168-7e9f-46e5-8dea-dd3ab884cd03 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:25.138731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:25.138836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-65b1e168-7e9f-46e5-8dea-dd3ab884cd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:25.147274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 226/429] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:24 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=4e8a255633994167b69cee0662997d5e => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:25.149249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:25.214690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 204/430] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:25.262305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1e168-7e9f-46e5-8dea-dd3ab884cd03 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:25.302006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0a977-685d-4143-8a18-8fbae228204f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:25.399921 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:25.400820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.401102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.401370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] PATH_INFO: `/v3/projects/c7555237bf4842dd8f28617418adec46` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.403865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1e168-7e9f-46e5-8dea-dd3ab884cd03 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:25.406246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] RBAC: Authorizing `identity:get_project(project_id=c7555237bf4842dd8f28617418adec46)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:25.407773 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:25.410887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 227/431] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:25 2022] GET /identity/v3/projects/c7555237bf4842dd8f28617418adec46 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:25.423667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:25.459389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:25.460598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.461142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.461614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.472433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:25.480702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:25.481669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 04 14:38:25.482472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:25.488959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 205/432] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:25.494537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:25.549137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97226-a5c0-4a88-8cc2-02c9e5ef570a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:25.596423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:25.598116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.598392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.598659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.599905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:25.601427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:25.603087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 228/433] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:25 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:25.609574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:25.640665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fadf4-fbd0-4755-8f97-ee4ad3fe54ee None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:25.701074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:25.701708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.701992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.702259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.703252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] RBAC: Adding query filter params (name=member) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:25.703356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:25.704805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:25.721013 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 206/434] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:38:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:25.722491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:25.752271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c141111-b33b-43a9-82ed-31812cdaba70 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:25.801414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e18a86-0ba1-459b-be0a-3ef72a742542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:25.802250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e18a86-0ba1-459b-be0a-3ef72a742542 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.802532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e18a86-0ba1-459b-be0a-3ef72a742542 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.803113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e18a86-0ba1-459b-be0a-3ef72a742542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.804313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e18a86-0ba1-459b-be0a-3ef72a742542 None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=c7555237bf4842dd8f28617418adec46) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:25.804617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e18a86-0ba1-459b-be0a-3ef72a742542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:25.806215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e18a86-0ba1-459b-be0a-3ef72a742542 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:25.813148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 229/435] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:25 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=c7555237bf4842dd8f28617418adec46&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:25.814582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027634-8e71-456c-90ff-fcc369644c0d None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.814873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027634-8e71-456c-90ff-fcc369644c0d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.815310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4027634-8e71-456c-90ff-fcc369644c0d None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.816854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 230/436] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:25.822548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bf662-2092-4dee-adf9-b2e55bcde2a0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.822912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bf662-2092-4dee-adf9-b2e55bcde2a0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.823814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bf662-2092-4dee-adf9-b2e55bcde2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.847665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0f9bf662-2092-4dee-adf9-b2e55bcde2a0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:25.849136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0f9bf662-2092-4dee-adf9-b2e55bcde2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:25.854521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ff4a4-66d1-4163-b6ef-a4441df4843a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:25.905352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:25.906407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:25.906907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:25.907394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:25.954888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 231/437] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:25.988509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bf662-2092-4dee-adf9-b2e55bcde2a0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.000127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.001653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.110027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 207/438] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:38:26.112043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bf662-2092-4dee-adf9-b2e55bcde2a0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.115886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.163768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db85de4-0549-490d-bb89-c2f329a56f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.164599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db85de4-0549-490d-bb89-c2f329a56f90 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.164941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db85de4-0549-490d-bb89-c2f329a56f90 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.165212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db85de4-0549-490d-bb89-c2f329a56f90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.177096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db85de4-0549-490d-bb89-c2f329a56f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-1db85de4-0549-490d-bb89-c2f329a56f90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:26.185698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:26.186680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:26.187489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 232/439] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:26.189720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712364d-c1ab-4109-872d-319a220cd4be None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.190002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712364d-c1ab-4109-872d-319a220cd4be None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.190272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712364d-c1ab-4109-872d-319a220cd4be None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.192026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 233/440] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:26.192828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712364d-c1ab-4109-872d-319a220cd4be None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.234872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c1050-2a9b-4cbd-a39c-a70e0e86b373 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.271395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdfb22-b192-446e-9ed9-c5f0a943374b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.272276 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfb22-b192-446e-9ed9-c5f0a943374b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.272946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfb22-b192-446e-9ed9-c5f0a943374b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.273225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfb22-b192-446e-9ed9-c5f0a943374b None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.274475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdfb22-b192-446e-9ed9-c5f0a943374b None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.276003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdfb22-b192-446e-9ed9-c5f0a943374b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.282508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 208/441] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:26.283921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8330-8472-4548-84e0-0a31b7d8febd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.284209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8330-8472-4548-84e0-0a31b7d8febd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.284562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c8330-8472-4548-84e0-0a31b7d8febd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.301581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-593c8330-8472-4548-84e0-0a31b7d8febd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:26.302775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-593c8330-8472-4548-84e0-0a31b7d8febd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:26.310267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712364d-c1ab-4109-872d-319a220cd4be None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.345131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.345834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.346110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.346382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.347107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:26.347392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.348837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.357369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 234/442] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:26.358175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.372137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 209/443] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:26.373710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8330-8472-4548-84e0-0a31b7d8febd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.453175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a029aa-9c3f-4f1c-a20e-96e27b193dab None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.471222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c8330-8472-4548-84e0-0a31b7d8febd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.489018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.489812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.490087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.490352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.491531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.493028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.494722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 235/444] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:26.495530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.506871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.507545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.507830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.508111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.584033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.585558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.597407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f875494-1496-4964-8eb8-aa4e463a0758 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.630851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.631680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.631942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.632215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.642275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:26.650405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:26.651489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:26.652657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 236/445] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:26.653610 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.690496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 210/446] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:26.691379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.754366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c6327-ea5d-461a-ae11-1421541bcf7d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.788722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.789409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.789537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75c78-958f-4699-bfc5-0e96ca8597f6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.789697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.789964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.790824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=4e8a255633994167b69cee0662997d5e, scope.project.id=cefff7c16392441d88d3526f6461ccb4) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:26.791131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.792569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.798372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 237/447] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=4e8a255633994167b69cee0662997d5e => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:26.799171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.825001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.825709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.825991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.826269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.827059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:26.827368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.828834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.838984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 211/448] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:26.845014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.925765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6229f24e-d512-41a3-87e7-579efc4e2cce None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.954980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d83d8-e58a-4a63-beaf-cdb431f4bc9d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:26.962373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.963408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.963689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.963961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] PATH_INFO: `/v3/projects/cefff7c16392441d88d3526f6461ccb4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:26.965336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] RBAC: Authorizing `identity:get_project(project_id=cefff7c16392441d88d3526f6461ccb4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:26.966885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:26.973026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 238/449] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/projects/cefff7c16392441d88d3526f6461ccb4 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:26.974692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:26.993109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:26.993880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:26.994171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:26.994444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:27.083095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff73f19-88df-4e56-ba77-c7f3b75d8021 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:27.085055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:27.086559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:27.124186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:27.125156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:27.125438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:27.126547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] PATH_INFO: `/v3/groups/353d3434c82e41859ed00edffbbf04d1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:27.127916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] RBAC: Authorizing `identity:get_group(group_id=353d3434c82e41859ed00edffbbf04d1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:27.128503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:27.130518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 239/450] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/groups/353d3434c82e41859ed00edffbbf04d1 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:27.136494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:27.205801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 212/451] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:38:27.211746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:27.251535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642929b-c6c3-4c05-aab5-decb4455ad78 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:27.283912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:27.284573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:27.284881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:27.285161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:27.285997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] RBAC: Adding query filter params (group.id=353d3434c82e41859ed00edffbbf04d1, role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=cefff7c16392441d88d3526f6461ccb4) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:27.286334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:27.287781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:27.293826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 240/452] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:27 2022] GET /identity/v3/role_assignments?group.id=353d3434c82e41859ed00edffbbf04d1&scope.project.id=cefff7c16392441d88d3526f6461ccb4&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:27.314787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79343b3d-14f3-4161-9135-a67fe9dfb6e9 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:27.353514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:27.354301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:27.354538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:27.354869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] PATH_INFO: `/v3/projects/62c598d6329d4180a1bbd38c5500b8cc` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:27.356267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] RBAC: Authorizing `identity:get_project(project_id=62c598d6329d4180a1bbd38c5500b8cc)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:27.358029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:27.359893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 213/453] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:38:27 2022] GET /identity/v3/projects/62c598d6329d4180a1bbd38c5500b8cc => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:27.366286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:27.495457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d26fea-b9c8-4fde-90a2-6038f0beb314 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:27.533125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00e6a3-84e5-4f38-8b4f-a573aa5e9291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:27.533985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e6a3-84e5-4f38-8b4f-a573aa5e9291 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:27.534263 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e6a3-84e5-4f38-8b4f-a573aa5e9291 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:27.534560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00e6a3-84e5-4f38-8b4f-a573aa5e9291 None admin] PATH_INFO: `/v3/groups/12f4d7fe42c84263949f3c0406ba1d76` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:27.535812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e6a3-84e5-4f38-8b4f-a573aa5e9291 None admin] RBAC: Authorizing `identity:get_group(group_id=12f4d7fe42c84263949f3c0406ba1d76)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:27.537348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00e6a3-84e5-4f38-8b4f-a573aa5e9291 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:27.539050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 241/454] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 4 14:38:27 2022] GET /identity/v3/groups/12f4d7fe42c84263949f3c0406ba1d76 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:27.545631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:27.653756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940367-e2b8-42c9-896e-415722336e05 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:27.687240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cc1d6-ed30-4b99-bcec-f0d67b180121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:27.687902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cc1d6-ed30-4b99-bcec-f0d67b180121 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:27.688181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cc1d6-ed30-4b99-bcec-f0d67b180121 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:27.688455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cc1d6-ed30-4b99-bcec-f0d67b180121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:27.689380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cc1d6-ed30-4b99-bcec-f0d67b180121 None admin] RBAC: Adding query filter params (group.id=12f4d7fe42c84263949f3c0406ba1d76, role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=62c598d6329d4180a1bbd38c5500b8cc) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:27.689710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cc1d6-ed30-4b99-bcec-f0d67b180121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:27.691132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cc1d6-ed30-4b99-bcec-f0d67b180121 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:27.697362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 214/455] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 4 14:38:27 2022] GET /identity/v3/role_assignments?group.id=12f4d7fe42c84263949f3c0406ba1d76&scope.project.id=62c598d6329d4180a1bbd38c5500b8cc&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:30.311405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12be53-7c7c-4e64-9d5b-6c0b53cd4066 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.311934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12be53-7c7c-4e64-9d5b-6c0b53cd4066 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.311934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12be53-7c7c-4e64-9d5b-6c0b53cd4066 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.313735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 242/456] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:30.319928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483a42-c4f4-4516-8714-8f3ee5126d79 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.320454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483a42-c4f4-4516-8714-8f3ee5126d79 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.320528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483a42-c4f4-4516-8714-8f3ee5126d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.349948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-67483a42-c4f4-4516-8714-8f3ee5126d79 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:30.351150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-67483a42-c4f4-4516-8714-8f3ee5126d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:30.450766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 215/457] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:30.493492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12be53-7c7c-4e64-9d5b-6c0b53cd4066 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:30.617658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae94e2-5720-477a-99e4-e8270f40db0c None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.618024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae94e2-5720-477a-99e4-e8270f40db0c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.618253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae94e2-5720-477a-99e4-e8270f40db0c None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.619903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 216/458] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:30.625672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11d44-d1a0-4a60-8082-dcd060750ea3 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.626258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11d44-d1a0-4a60-8082-dcd060750ea3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.626727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11d44-d1a0-4a60-8082-dcd060750ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.638054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12be53-7c7c-4e64-9d5b-6c0b53cd4066 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:30.646372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-82a11d44-d1a0-4a60-8082-dcd060750ea3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:30.648207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-82a11d44-d1a0-4a60-8082-dcd060750ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:30.705418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:30.706233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.706511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.706783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.733372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 217/459] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:30.737939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:30.739424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:30.753242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a11d44-d1a0-4a60-8082-dcd060750ea3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:30.763728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:30.764659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:30.765554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:30.765554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:30.765554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:30.765554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:30.772939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 243/460] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:30.778727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:30.867608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a11d44-d1a0-4a60-8082-dcd060750ea3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:30.911159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95407-2eba-4865-8988-5dd73b88ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:30.912876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95407-2eba-4865-8988-5dd73b88ac6d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.913561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95407-2eba-4865-8988-5dd73b88ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.913986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95407-2eba-4865-8988-5dd73b88ac6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.928683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95407-2eba-4865-8988-5dd73b88ac6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:30.930440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95407-2eba-4865-8988-5dd73b88ac6d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:30.930628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473096f-8b04-4310-a78a-aa74f5ea645c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-10e95407-2eba-4865-8988-5dd73b88ac6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:30.956034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:30.957231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:30.958301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:30.958301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:30.958301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:30.958301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:30.965547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 218/461] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:30.965547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e019b-6954-453a-895e-7d9e80503574 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.965547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e019b-6954-453a-895e-7d9e80503574 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.965547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e019b-6954-453a-895e-7d9e80503574 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.969300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 219/462] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:30.970780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e019b-6954-453a-895e-7d9e80503574 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:30.974169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e79159-eaeb-4196-8f69-3c399bd83ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:30.974875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79159-eaeb-4196-8f69-3c399bd83ebc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.975150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79159-eaeb-4196-8f69-3c399bd83ebc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.975424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e79159-eaeb-4196-8f69-3c399bd83ebc None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:30.976424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e79159-eaeb-4196-8f69-3c399bd83ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:30.976533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e79159-eaeb-4196-8f69-3c399bd83ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:30.977910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e79159-eaeb-4196-8f69-3c399bd83ebc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:30.995655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 244/463] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:30.997109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902cf6d-fd49-4646-b76a-8d708fa7eb3b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:30.997401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902cf6d-fd49-4646-b76a-8d708fa7eb3b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:30.997724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902cf6d-fd49-4646-b76a-8d708fa7eb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.016901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e902cf6d-fd49-4646-b76a-8d708fa7eb3b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:31.018217 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e902cf6d-fd49-4646-b76a-8d708fa7eb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:31.098887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e019b-6954-453a-895e-7d9e80503574 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:31.102712 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 245/464] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:31.103629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902cf6d-fd49-4646-b76a-8d708fa7eb3b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:31.155529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:31.156624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.157146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.157746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.159519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:31.160030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:31.161745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:31.182325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 220/465] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:31.182325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:31.237685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902cf6d-fd49-4646-b76a-8d708fa7eb3b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:31.270272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:31.270953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.271232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.271512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.272227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:31.274068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:31.291393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80593ac3-5826-4172-8d2b-9364caeebecf None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:31.304999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 246/466] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:31 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:31.304999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:31.338896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57fb59-0891-4dc8-ab0a-990507634c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:31.340479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57fb59-0891-4dc8-ab0a-990507634c2b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.340994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57fb59-0891-4dc8-ab0a-990507634c2b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.341488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57fb59-0891-4dc8-ab0a-990507634c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.354795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57fb59-0891-4dc8-ab0a-990507634c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:31.356693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57fb59-0891-4dc8-ab0a-990507634c2b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-cb57fb59-0891-4dc8-ab0a-990507634c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:31.376389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:31.377379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:31.378236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:31.378236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:31.378236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:31.378236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:31.380720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 221/467] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:31.382461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28292-de49-47f8-baba-715f5cf49364 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.382969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28292-de49-47f8-baba-715f5cf49364 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.383431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc28292-de49-47f8-baba-715f5cf49364 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.389219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 222/468] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:31.390478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc28292-de49-47f8-baba-715f5cf49364 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:31.428095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3706bf2-d0f4-4378-9533-0e82694f8431 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:31.464333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e4f28-e654-4161-9db1-d04b27a763b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:31.464998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e4f28-e654-4161-9db1-d04b27a763b1 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.465287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e4f28-e654-4161-9db1-d04b27a763b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.465563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e4f28-e654-4161-9db1-d04b27a763b1 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.466294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e4f28-e654-4161-9db1-d04b27a763b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:31.467800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e4f28-e654-4161-9db1-d04b27a763b1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:31.494594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 247/469] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:31 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:31.497613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5c9ee-10ac-4993-8e61-fa40ddda83a4 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.501170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5c9ee-10ac-4993-8e61-fa40ddda83a4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.502731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5c9ee-10ac-4993-8e61-fa40ddda83a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.523283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc28292-de49-47f8-baba-715f5cf49364 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:31.528507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c2d5c9ee-10ac-4993-8e61-fa40ddda83a4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:31.530092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c2d5c9ee-10ac-4993-8e61-fa40ddda83a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:31.571632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:31.572394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.572676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.573003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.573803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:31.574152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:31.575653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:31.596889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 223/470] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:31.599612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:31.614556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 248/471] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:31.641772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5c9ee-10ac-4993-8e61-fa40ddda83a4 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:31.730680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea501c4b-2ae0-4d8f-bfcf-a621f7b8c064 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:31.757306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5c9ee-10ac-4993-8e61-fa40ddda83a4 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:31.780713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:31.781512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.781794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.782066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.783482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:31.785016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:31.805285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:31.807158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:31.807476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:31.807759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:31.816006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 224/472] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:31 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:31.825076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:31.826766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:31.847889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:31.848895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:31.849792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:31.849792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:31.849792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:31.849792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:31.850442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 249/473] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:31.856077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:31.968292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a7cb-608f-4240-8373-97a2938b10b2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:32.005160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf106a47-d63e-4b7c-9504-2c1074bb48a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:32.006105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf106a47-d63e-4b7c-9504-2c1074bb48a6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:32.006532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf106a47-d63e-4b7c-9504-2c1074bb48a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:32.006954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf106a47-d63e-4b7c-9504-2c1074bb48a6 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:32.007881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf106a47-d63e-4b7c-9504-2c1074bb48a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:32.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf106a47-d63e-4b7c-9504-2c1074bb48a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:32.010000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf106a47-d63e-4b7c-9504-2c1074bb48a6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:32.024229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 225/474] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:32.029638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:32.111769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dfa01-2adb-4fa2-92bf-696e185c01eb None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:32.112450 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dfa01-2adb-4fa2-92bf-696e185c01eb None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:32.112954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dfa01-2adb-4fa2-92bf-696e185c01eb None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:32.114801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 226/475] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:32.126958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef128-5b98-43f6-bc9a-f655ec515754 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:32.127276 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef128-5b98-43f6-bc9a-f655ec515754 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:32.127462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef128-5b98-43f6-bc9a-f655ec515754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:32.149698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-96cef128-5b98-43f6-bc9a-f655ec515754 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:32.151071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-96cef128-5b98-43f6-bc9a-f655ec515754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:32.152207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98717ab-7f91-45e4-8c06-4a85501f8f69 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:32.192378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:32.193439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:32.193920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:32.194391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:32.195347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:32.197535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:32.225000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 250/476] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:32 2022] POST /identity/v3/users => generated 249 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:32.232622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 227/477] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:32.251220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:32.370846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f011db-d770-4b26-99fc-36de910cc421 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:32.408756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:32.409541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:32.409885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:32.410162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:32.411014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:32.412446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:32.420310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.sql.core [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 04 14:38:32.420310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 04 14:38:32.420310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [parameters: {'id': 'e840201eb32847c8b0877af910a0fd7d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 04 14:38:32.420310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91630) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result = self._query(query) May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application conn.query(q) May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result.read() May 04 14:38:32.440038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application packet.raise_for_error() May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return ref.to_dict() May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 04 14:38:32.441421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application next(self.gen) May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application yield resource May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application next(self.gen) May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.session.rollback() May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.force_reraise() May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise self.value May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application session.commit() May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 04 14:38:32.442328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._prepare_impl() May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.session.flush() May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._flush(objects) May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application compat.raise_( May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application flush_context.execute() May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rec.execute(self) May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 04 14:38:32.443181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application _emit_insert_statements( May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application c = connection._execute_20( May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ret = self._execute_context( May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 04 14:38:32.444038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result = self._query(query) May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application conn.query(q) May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result.read() May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 04 14:38:32.444867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application packet.raise_for_error() May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [parameters: {'id': 'e840201eb32847c8b0877af910a0fd7d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:32.445657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 04 14:38:32.446482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:32.447430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 251/478] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 4 14:38:32 2022] POST /identity/v3/roles => generated 144 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:32.447830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef128-5b98-43f6-bc9a-f655ec515754 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:32.566936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef128-5b98-43f6-bc9a-f655ec515754 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:32.605901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218d592-dd82-404f-b4e0-f637478b76ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:32.606709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218d592-dd82-404f-b4e0-f637478b76ad None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:32.607000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218d592-dd82-404f-b4e0-f637478b76ad None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:32.607267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218d592-dd82-404f-b4e0-f637478b76ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:32.618828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218d592-dd82-404f-b4e0-f637478b76ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-6218d592-dd82-404f-b4e0-f637478b76ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:32.627307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:32.628186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 04 14:38:32.629034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:32.632941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 228/479] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:38:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:32.635245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:32.742215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450b4da-2a95-4429-9f37-5cae06314a96 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:32.780003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7b6ac-7d07-487f-944d-a0a67191a0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:32.780692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b6ac-7d07-487f-944d-a0a67191a0d4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:32.781005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b6ac-7d07-487f-944d-a0a67191a0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:32.781334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b6ac-7d07-487f-944d-a0a67191a0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:32.782113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7b6ac-7d07-487f-944d-a0a67191a0d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:32.782420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7b6ac-7d07-487f-944d-a0a67191a0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:32.783872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7b6ac-7d07-487f-944d-a0a67191a0d4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:32.796936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 252/480] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:34.223939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0217baa-138c-4dd2-a32f-16227e200c55 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.224536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0217baa-138c-4dd2-a32f-16227e200c55 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.224536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0217baa-138c-4dd2-a32f-16227e200c55 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.226180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 229/481] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:34.233350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77d744-057d-48e4-b8b3-836fda6690fd None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.233868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77d744-057d-48e4-b8b3-836fda6690fd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.233868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77d744-057d-48e4-b8b3-836fda6690fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.257868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ad77d744-057d-48e4-b8b3-836fda6690fd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:34.259069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ad77d744-057d-48e4-b8b3-836fda6690fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:34.351029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 253/482] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:34.372069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f70421-3315-473f-951e-2babf937c004 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.372710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f70421-3315-473f-951e-2babf937c004 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.373212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f70421-3315-473f-951e-2babf937c004 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.375810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 230/483] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:34.380052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055d21-d48b-4307-bfc4-b7c18af9fb10 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.380336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055d21-d48b-4307-bfc4-b7c18af9fb10 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.380612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055d21-d48b-4307-bfc4-b7c18af9fb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.383365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f70421-3315-473f-951e-2babf937c004 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:34.398685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-23055d21-d48b-4307-bfc4-b7c18af9fb10 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:34.400335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-23055d21-d48b-4307-bfc4-b7c18af9fb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:34.467390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 254/484] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:34.490738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055d21-d48b-4307-bfc4-b7c18af9fb10 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:34.533569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f70421-3315-473f-951e-2babf937c004 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:34.593211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055d21-d48b-4307-bfc4-b7c18af9fb10 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:34.593525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:34.594557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.595007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.595407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.614826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:34.627332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:34.628369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:34.629532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 231/485] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:34.636720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:34.637671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.638006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.638336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.645854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:34.650906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:34.660769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:34.663490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:34.665188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 255/486] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:34.674032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:34.760673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0a42e-b761-4f4e-a587-8a352cbcc177 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:34.799185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:34.799859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.800138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.800413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.801213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:34.801508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:34.802889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:34.817950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 232/487] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:34.824264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:34.877063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da34a8-7ec6-4d25-9462-01803c52c340 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:34.948467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cecb3-7aa0-476d-9eb1-cf7f47c37374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:34.949594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cecb3-7aa0-476d-9eb1-cf7f47c37374 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.950385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cecb3-7aa0-476d-9eb1-cf7f47c37374 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.950872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cecb3-7aa0-476d-9eb1-cf7f47c37374 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.953441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7cecb3-7aa0-476d-9eb1-cf7f47c37374 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:34.953845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7cecb3-7aa0-476d-9eb1-cf7f47c37374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:34.957052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7cecb3-7aa0-476d-9eb1-cf7f47c37374 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:34.963957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2758d-fd74-4a4d-bfc3-07bb2a5eb1bf None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:34.977005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 256/488] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:34.980160 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606d938-bd25-4772-ab71-b6b9d4b2fdfb None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:34.980611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606d938-bd25-4772-ab71-b6b9d4b2fdfb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:34.981080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606d938-bd25-4772-ab71-b6b9d4b2fdfb None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:34.985216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 257/489] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:34.986306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606d938-bd25-4772-ab71-b6b9d4b2fdfb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.005780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024e6da-6e32-46ae-beb3-5cebce139838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.007059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e6da-6e32-46ae-beb3-5cebce139838 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.007554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e6da-6e32-46ae-beb3-5cebce139838 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.008039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e6da-6e32-46ae-beb3-5cebce139838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.025964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024e6da-6e32-46ae-beb3-5cebce139838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.029005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024e6da-6e32-46ae-beb3-5cebce139838 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-5024e6da-6e32-46ae-beb3-5cebce139838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:35.056220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:35.057548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:35.058811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:35.058811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:35.058811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:35.058811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:35.065378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 233/490] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:35.065378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f582b-1aa5-45ec-bcdd-0f7792f77317 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.065378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f582b-1aa5-45ec-bcdd-0f7792f77317 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.065378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f582b-1aa5-45ec-bcdd-0f7792f77317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.085918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-882f582b-1aa5-45ec-bcdd-0f7792f77317 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:35.087135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-882f582b-1aa5-45ec-bcdd-0f7792f77317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:35.128712 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606d938-bd25-4772-ab71-b6b9d4b2fdfb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.163620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf6fa6-d3c1-4782-9b61-8be3e63525ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.164423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6fa6-d3c1-4782-9b61-8be3e63525ca None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.164699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6fa6-d3c1-4782-9b61-8be3e63525ca None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.165024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6fa6-d3c1-4782-9b61-8be3e63525ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.165965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 234/491] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:35.167466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f582b-1aa5-45ec-bcdd-0f7792f77317 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.176393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf6fa6-d3c1-4782-9b61-8be3e63525ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.177900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf6fa6-d3c1-4782-9b61-8be3e63525ca None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-35cf6fa6-d3c1-4782-9b61-8be3e63525ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:35.195819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:35.196812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:35.197706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:35.197706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:35.197706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:35.197706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:35.201916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 258/492] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:35.201916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca9ba-3748-424b-aea6-127130357ca9 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.201916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca9ba-3748-424b-aea6-127130357ca9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.201916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ca9ba-3748-424b-aea6-127130357ca9 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.202441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 259/493] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:35.203317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ca9ba-3748-424b-aea6-127130357ca9 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.317138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ca9ba-3748-424b-aea6-127130357ca9 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.332404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f582b-1aa5-45ec-bcdd-0f7792f77317 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.355917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.356721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.357036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.357309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.368210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:35.368402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4584b-8dc2-41bf-bedf-8433f1ab4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.368998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4584b-8dc2-41bf-bedf-8433f1ab4681 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.369280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4584b-8dc2-41bf-bedf-8433f1ab4681 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.369556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4584b-8dc2-41bf-bedf-8433f1ab4681 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.370306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4584b-8dc2-41bf-bedf-8433f1ab4681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:35.370612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4584b-8dc2-41bf-bedf-8433f1ab4681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.372007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4584b-8dc2-41bf-bedf-8433f1ab4681 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:35.377281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:35.378385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:35.379373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:35.381465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 260/494] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:35.382456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.390803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 235/495] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:35.392062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b9e4-b71f-4256-b0d6-887872a7d18e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.392354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b9e4-b71f-4256-b0d6-887872a7d18e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.392632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d2b9e4-b71f-4256-b0d6-887872a7d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.411858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-29d2b9e4-b71f-4256-b0d6-887872a7d18e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:35.413068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-29d2b9e4-b71f-4256-b0d6-887872a7d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:35.482366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 236/496] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:35.483338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b9e4-b71f-4256-b0d6-887872a7d18e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.491513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c261feb-9b35-423c-ae57-0a29e0f8d9e3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.527097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.527758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.528042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.528316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.529068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:35.529376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.530742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.543511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 261/497] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:35.544270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.590077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d2b9e4-b71f-4256-b0d6-887872a7d18e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.626954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.627630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.627910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.628181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.628963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:35.629267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.630631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.644222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 237/498] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:35.644222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.648755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca4c2b-861e-42ad-8d01-f8a09ef80cee None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.684424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.685261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.685538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.685801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.695019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.696476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:35.712366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:35.713564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 262/499] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:35.714466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.752778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566bfc-432e-4180-98ff-83ba9ed2c2c6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.786716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.787524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.787807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.788081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.798489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:35.806262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:35.807380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:35.808221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 238/500] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:35.808918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.824205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133d0b9-e641-437c-9b35-e49319e00048 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.857542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.858259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.858540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.858812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] PATH_INFO: `/v3/users/glance` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.865806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.867241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:35.881486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:35.882453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:35.883342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 263/501] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:35.883902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.909618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86350665-6523-42d7-b19f-af2549b911d8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:35.941992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:35.942815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:35.943120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:35.943383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:35.954110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:35.955571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:35.972485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:35.973548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:35.974427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:35.974427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:35.974427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:35.974427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:35.974427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 239/502] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:35.974703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:35.981678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76f38f-a99d-4f36-87ce-c71364497f8e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.016400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d0c30-e2fb-4c6e-a205-8bbe73a5ec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.017089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0c30-e2fb-4c6e-a205-8bbe73a5ec78 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.017373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0c30-e2fb-4c6e-a205-8bbe73a5ec78 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.017641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d0c30-e2fb-4c6e-a205-8bbe73a5ec78 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.018490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d0c30-e2fb-4c6e-a205-8bbe73a5ec78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:36.018796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d0c30-e2fb-4c6e-a205-8bbe73a5ec78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.020221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d0c30-e2fb-4c6e-a205-8bbe73a5ec78 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.057533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 264/503] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:36.058869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd795-5670-4fdd-971f-52800d5d50bd None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.059127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd795-5670-4fdd-971f-52800d5d50bd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.059426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd795-5670-4fdd-971f-52800d5d50bd None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.060844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 265/504] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:36.061351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfd795-5670-4fdd-971f-52800d5d50bd None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.080946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e83a9-dc1e-44e3-b634-6d7e16e81b69 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.114441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.115017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.115294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.115563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.116305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:36.116606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.118008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.125959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 240/505] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:36.126744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.162821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfd795-5670-4fdd-971f-52800d5d50bd None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.196555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.197300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.197573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.197839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.198672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:36.198978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.200427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.221616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 266/506] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:36.222474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.235264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e1dd3-f867-49b6-b256-af26df676987 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.267470 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04654f94-35eb-4aba-ba52-99d1d1cab555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.268158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04654f94-35eb-4aba-ba52-99d1d1cab555 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.268441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04654f94-35eb-4aba-ba52-99d1d1cab555 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.268705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04654f94-35eb-4aba-ba52-99d1d1cab555 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.269455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04654f94-35eb-4aba-ba52-99d1d1cab555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:36.269754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04654f94-35eb-4aba-ba52-99d1d1cab555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.271138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04654f94-35eb-4aba-ba52-99d1d1cab555 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.283725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 241/507] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:36.284998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e2408-951d-4006-8cfb-53aa89c31ca7 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.285258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e2408-951d-4006-8cfb-53aa89c31ca7 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.285531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e2408-951d-4006-8cfb-53aa89c31ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.302139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c87e2408-951d-4006-8cfb-53aa89c31ca7 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:36.303338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c87e2408-951d-4006-8cfb-53aa89c31ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:36.328037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b008b6-2a52-4bed-b210-c546f6fd1bfd None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.361351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.361996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.362272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.362541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.370779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 242/508] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:36.371618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e2408-951d-4006-8cfb-53aa89c31ca7 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.430504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.432069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.465720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e2408-951d-4006-8cfb-53aa89c31ca7 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.500716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.501532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.501808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.502071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.512732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.514271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.526675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 267/509] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:36.527535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:36.532471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:36.533396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:36.534350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:36.534350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:36.534350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:36.534350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:36.534350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 243/510] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:36.534624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.633100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa7df4-5c8f-4777-ac6a-ea7378730eb9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.650978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162c51-b50c-4205-ae1b-39b7c1e5a458 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.667959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.668617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.668922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.669191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.689814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.690621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.690906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.691181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.698719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.700186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:36.716862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:36.717971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:36.718860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 244/511] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:36.719440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.738962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.740431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.816543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51f60f-58aa-47bb-a71b-4a9f9b368242 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.838369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 268/512] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:36.839228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.854883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.855900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.856239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.856597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.868345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.869949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:36.887867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:36.888767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:36.889692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:36.889692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:36.889692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:36.889692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:36.889692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 245/513] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:36.890031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:36.938009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff665b91-8840-4f8a-b116-acb5ab9a715b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.973445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:36.974725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:36.975204 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:36.975484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:36.989269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:36.989516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5acc67-c399-4af4-920d-a5e054f2fa9f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:36.990756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:37.008342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:37.009482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:37.010361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:37.010361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:37.010361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:37.010361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:37.013169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 269/514] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:37.013896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.023562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.024239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.024527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.024867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.025617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:37.025921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.027313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.039214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 246/515] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:37.040103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.117131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e964cd-7c9d-4ccd-8770-6497fe96c153 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.146043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a551753-6afa-4671-9d40-1bae403ab385 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.154171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.154997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.155308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.155595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.156459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:37.156768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.158269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.177272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 270/516] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:37.178181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.181747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.182580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.182858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.183123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.194736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.196207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:37.214034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:37.215114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:37.215953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:37.215953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:37.215953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:37.215953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:37.215953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 247/517] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:37.216230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.280928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d05e6-61dc-48fc-bbbf-b22e935f0ee8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.321275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e45da-ef88-433a-be84-d3dfe656677c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.323971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.324682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.324993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.325268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.326011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:37.326315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.327700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.339934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 271/518] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:37.340829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.358111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.358793 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.359072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.359336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.360059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:37.360359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.361779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.373667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 248/519] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:37.374505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.439293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874211af-ffd7-418c-9cf6-00f25b3524a7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.471451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989f23b-75a5-4484-8784-882dc3414984 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.473017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.473827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.474102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.474372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.481973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.483509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:37.497674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:37.498781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 272/520] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:37.499684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.506655 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.507313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.507592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.507862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.571379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.572938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.595825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97679d-9558-4df8-8c65-f9433c92d32b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.636068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.636738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.637049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.637319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.638045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:37.638350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.639733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.653214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 273/521] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:37.653709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.674688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 249/522] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:37.675541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.750417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af0a1-f835-4611-9dc0-8c212578f2ff None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.768405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d24010-1872-4385-abb5-a8f9b2b4e577 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.783299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.783990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.784272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.784546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.785324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.786806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.802757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.803606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.803921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.804210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.807638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 274/523] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:37 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:37.808463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.811273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.812819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:37.828675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:37.829661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:37.830556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:37.835090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 250/524] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:37.835881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.918114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7713d-cfc9-4c77-89e8-a2d55fb446f1 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.942626 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320feac-015f-4dcc-a4ed-9f28205b5c9e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:37.954621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.955365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.955645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.955943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.956822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:37.957159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:37.958665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:37.983931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:37.984937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:37.985341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:37.985711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:37.991000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 275/525] 10.222.0.32 () {56 vars in 1139 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:37.991000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:37.998882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.000499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:38.015303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:38.016215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 251/526] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:38.018634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.091404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da71773-a2e3-48f9-a7b9-b0647f9f7d24 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.116908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187990-f561-466b-92ea-31687a145e48 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.123872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.124699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.125011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.125278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.135518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.137020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.150077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.150737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.151015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.151286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.152073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:38.152381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.153836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:38.154837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:38.155666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:38.156537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:38.156537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:38.156537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:38.156537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:38.160589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 276/527] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:38.161489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.173883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 252/528] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:38.175143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.264163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a02ad-cb44-41a7-8590-af7214a0698c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.276059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf684b33-ef5a-451a-bf55-3950201579c3 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.298709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.299464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.299835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.300174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.313194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.314159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.314667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.315140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.316670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:38.317124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.320198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.338519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 253/529] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:38.339573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.371212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.372704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.435113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09b4a4-1cde-4e4b-95c1-22d7a2566134 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.464672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 277/530] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:38.469219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.471864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.472542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.472844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.473127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.473859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:38.474162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.475529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.487320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 254/531] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:38.488367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.559860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facca06-1aba-4229-a3d0-c16887d27a1b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.582104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f800b6-f76c-4bb6-be3a-890ed9fb20a2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.595599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.596257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.596528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.596811 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.597662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=753f2914e5bc4e298b99529e230d9ee3) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:38.598075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.599455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.613156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 278/532] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/role_assignments?user.id=753f2914e5bc4e298b99529e230d9ee3&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:38.613156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.626694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.627421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.627703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.627976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.628907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=fd3eb9e77dc64869b03b1190cf30e54b) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:38.629223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.630763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.637173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 255/533] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/role_assignments?user.id=fd3eb9e77dc64869b03b1190cf30e54b&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:38.638099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.721474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034f528-e94e-452f-9817-acf589f692ec None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.762728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.763577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.763858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.764128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.776623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.778217 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.793289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42655df6-6ef1-4211-93be-2ac7629a13b6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:38.797849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:38.798869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:38.800324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:38.800324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:38.800324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:38.800324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:38.800324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 279/534] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:38.810225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:38.860215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:38.861205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:38.861718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:38.863180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:38.871607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:38.873756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:38.894085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:38.895328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:38.896232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 256/535] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:38.904825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:39.003085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c09d6-0735-4b8e-9d29-f121bb2dc8bc None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:39.072521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:39.073137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:39.073411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:39.073679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:39.074409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:39.074707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:39.076081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:39.099790 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 280/536] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:39.106064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49517d-d87a-4d24-9fd4-dcfdb0194082 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:39.122511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:39.177299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:39.180340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:39.180777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:39.181486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:39.184918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:39.187125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:39.188737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:39.212675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 257/537] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:39.218549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:39.330992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276223df-2207-4e4b-93c9-a33749aab841 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:39.358965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f134bf8-42b2-4463-8ec6-ea12b685bf6e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:39.374128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:39.374791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:39.375093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:39.375369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:39.376200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=a4aed1720aea4ffa975f7936a1bdb3c1) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:39.376514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:39.377968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:39.389405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 258/538] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:39 2022] GET /identity/v3/role_assignments?user.id=a4aed1720aea4ffa975f7936a1bdb3c1&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:39.416656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:39.417510 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:39.417790 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:39.418056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:39.425667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:39.427196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:39.452988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:39.453927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:39.455253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 281/539] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:39.460534 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:39.571545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430c8bb-4440-4d54-91bd-688b18f0c669 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:39.608620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3cefb-c12a-42a5-97aa-67f19c0573ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:39.609291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3cefb-c12a-42a5-97aa-67f19c0573ea None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:39.609568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3cefb-c12a-42a5-97aa-67f19c0573ea None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:39.609846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3cefb-c12a-42a5-97aa-67f19c0573ea None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:39.610645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3cefb-c12a-42a5-97aa-67f19c0573ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:39.610952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3cefb-c12a-42a5-97aa-67f19c0573ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:39.612391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3cefb-c12a-42a5-97aa-67f19c0573ea None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:39.626017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 259/540] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:39.632185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:39.743238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a27dbc-e096-4a51-a799-d15b8fb11fb9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:39.778287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f408c73-17d9-4374-ba24-f6c152669955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:39.778949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408c73-17d9-4374-ba24-f6c152669955 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:39.779227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408c73-17d9-4374-ba24-f6c152669955 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:39.779496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408c73-17d9-4374-ba24-f6c152669955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:39.780346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f408c73-17d9-4374-ba24-f6c152669955 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=0d678d5ee7d74896bbc1316b32ec81ee) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:39.780654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f408c73-17d9-4374-ba24-f6c152669955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:39.782162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f408c73-17d9-4374-ba24-f6c152669955 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:39.788635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 282/541] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:39 2022] GET /identity/v3/role_assignments?user.id=0d678d5ee7d74896bbc1316b32ec81ee&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:40.282321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23614b-0ed6-4995-a9d1-192bf3017b58 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:40.282839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23614b-0ed6-4995-a9d1-192bf3017b58 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:40.283347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23614b-0ed6-4995-a9d1-192bf3017b58 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:40.286070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 260/542] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:40.294941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0504f8-2e74-406a-b652-980327122210 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:40.295278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0504f8-2e74-406a-b652-980327122210 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:40.295778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0504f8-2e74-406a-b652-980327122210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:40.329887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-af0504f8-2e74-406a-b652-980327122210 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:40.331141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-af0504f8-2e74-406a-b652-980327122210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:40.433138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 283/543] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:40.468161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23614b-0ed6-4995-a9d1-192bf3017b58 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:40.594382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23614b-0ed6-4995-a9d1-192bf3017b58 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:40.628361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:40.629195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:40.629472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:40.629740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:40.640343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:40.648345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:40.650551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:40.651657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 261/544] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:40.656226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0504f8-2e74-406a-b652-980327122210 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:40.815786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0504f8-2e74-406a-b652-980327122210 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:40.870694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:40.871737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:40.872233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:40.872714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:40.873709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:40.874213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:40.875790 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:40.888713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 284/545] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:40.896155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:41.055284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111648c0-0adb-4896-9dd4-6066312fa5df None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:41.109243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:41.110086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.110367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.110646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.125956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:41.127399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:41.148431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:41.149434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:41.151000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:41.151000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:41.151000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:41.151000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:41.151000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 262/546] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:41.155480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:41.299778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce1641-af1a-4758-ab44-8c0dd92c3c45 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:41.362303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44b0b3-474a-43c2-9c28-120fab509451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:41.363362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44b0b3-474a-43c2-9c28-120fab509451 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.363839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44b0b3-474a-43c2-9c28-120fab509451 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.364304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44b0b3-474a-43c2-9c28-120fab509451 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.365337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44b0b3-474a-43c2-9c28-120fab509451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:41.365837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44b0b3-474a-43c2-9c28-120fab509451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:41.367414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44b0b3-474a-43c2-9c28-120fab509451 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:41.384607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 285/547] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:41.389654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:41.573244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8f980-b7fd-4253-8af3-b3604175a284 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:41.624539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0d0f-d747-448c-bebc-839762922dd4 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.625550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0d0f-d747-448c-bebc-839762922dd4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.626028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0d0f-d747-448c-bebc-839762922dd4 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.629168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 286/548] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:41.632020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4cb58-b3cc-46e4-9cb9-f9263167c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:41.632020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4cb58-b3cc-46e4-9cb9-f9263167c8be None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.632020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4cb58-b3cc-46e4-9cb9-f9263167c8be None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.632020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4cb58-b3cc-46e4-9cb9-f9263167c8be None admin] PATH_INFO: `/v3/users/swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.634442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3a66-7354-4e23-aff6-08e4ced638ac None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.635039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3a66-7354-4e23-aff6-08e4ced638ac None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.635508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c3a66-7354-4e23-aff6-08e4ced638ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.641363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4cb58-b3cc-46e4-9cb9-f9263167c8be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:41.642844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4cb58-b3cc-46e4-9cb9-f9263167c8be None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:41.653529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ab0c3a66-7354-4e23-aff6-08e4ced638ac None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:41.655054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ab0c3a66-7354-4e23-aff6-08e4ced638ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-b8d4cb58-b3cc-46e4-9cb9-f9263167c8be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:41.658461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:41.659345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:41.660677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 263/549] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:41.661492 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aa5f1-01c2-4631-8cd2-50382afbddca None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.661743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aa5f1-01c2-4631-8cd2-50382afbddca None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.661954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aa5f1-01c2-4631-8cd2-50382afbddca None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.663500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 264/550] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:41.670062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01410-a5f1-4887-8899-2629c972806a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.670337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01410-a5f1-4887-8899-2629c972806a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.670616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed01410-a5f1-4887-8899-2629c972806a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.687576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5ed01410-a5f1-4887-8899-2629c972806a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:41.688627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5ed01410-a5f1-4887-8899-2629c972806a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:41.747544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 287/551] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:41.748922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3a66-7354-4e23-aff6-08e4ced638ac None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:41.772891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 265/552] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:41.797537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed01410-a5f1-4887-8899-2629c972806a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:41.863215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c3a66-7354-4e23-aff6-08e4ced638ac None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:41.906701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:41.907424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.907702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.908135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.909554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:41.909990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:41.912039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:41.918227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed01410-a5f1-4887-8899-2629c972806a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:41.936905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 288/553] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 4 14:38:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:41.937936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:41.966660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:41.967665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:41.968147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:41.969446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:41.983801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:41.991964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:41.993000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:41.993892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:41.998521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 266/554] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:41.999536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.058482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c4e47-1eb0-4f57-9dbb-49e5d494c119 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.094413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.095381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.095661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.095938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.103387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c41e2d-d503-44a5-8f02-a91b30a42d74 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.107408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:42.116023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:42.119388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 289/555] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:42.129062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.142828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700de8f5-0ecc-47d2-8f54-44912f218c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.143486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de8f5-0ecc-47d2-8f54-44912f218c25 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.143764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de8f5-0ecc-47d2-8f54-44912f218c25 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.144036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700de8f5-0ecc-47d2-8f54-44912f218c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.213821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700de8f5-0ecc-47d2-8f54-44912f218c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.215367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700de8f5-0ecc-47d2-8f54-44912f218c25 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.228644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5ee1-e4b4-40ec-8756-dc11ee2e6277 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.262186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.262864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.263171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.263424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.264189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:42.264496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.265907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.274749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 290/556] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:42.275610 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.334406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 267/557] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:42.335885 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87005c32-8230-4764-a4e7-df87ea2f6271 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.336233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87005c32-8230-4764-a4e7-df87ea2f6271 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.336573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87005c32-8230-4764-a4e7-df87ea2f6271 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.338020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 268/558] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:42.342918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87005c32-8230-4764-a4e7-df87ea2f6271 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.392562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0c7f9-3f53-4c83-98ab-ae9ce6d38925 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.427847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.428517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.428824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.429090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.429841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:42.430152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.431600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.444114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 291/559] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:42.444114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.454512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87005c32-8230-4764-a4e7-df87ea2f6271 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.490362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce820d-3c1c-482e-8b2a-8ba97789fb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.491100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce820d-3c1c-482e-8b2a-8ba97789fb64 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.491373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce820d-3c1c-482e-8b2a-8ba97789fb64 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.491648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce820d-3c1c-482e-8b2a-8ba97789fb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.503433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce820d-3c1c-482e-8b2a-8ba97789fb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.505293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce820d-3c1c-482e-8b2a-8ba97789fb64 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-1bce820d-3c1c-482e-8b2a-8ba97789fb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:42.525169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:42.526109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:42.526948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:42.526948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:42.526948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:42.526948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:42.526948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 269/560] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:42.527632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e433-fb62-4753-a9e3-2e9aa7e5d853 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.528216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e433-fb62-4753-a9e3-2e9aa7e5d853 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.528216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e433-fb62-4753-a9e3-2e9aa7e5d853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.546079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0e62e433-fb62-4753-a9e3-2e9aa7e5d853 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:42.547182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf792c2-31a8-4636-90f4-d3ac6fdb4d34 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.547294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0e62e433-fb62-4753-a9e3-2e9aa7e5d853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:42.580345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.581191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.581467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.581733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.593809 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.595547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:42.615117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:42.616871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:42.618542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:42.618542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:42.618542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:42.618542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:42.618542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 292/561] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:42.618542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.625439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 270/562] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:42.629683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e433-fb62-4753-a9e3-2e9aa7e5d853 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.725178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee0629-970a-4bf7-937a-ce5490127c84 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.759339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.760180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.760462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.760731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.765364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e433-fb62-4753-a9e3-2e9aa7e5d853 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.771481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.773366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:42.790289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:42.791723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:42.792826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:42.792826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:42.792826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:42.792826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:42.792826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 293/563] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:42.792826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.801000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.801676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.801952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.802222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.802961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:42.803266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.804630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.815921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 271/564] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:42.816731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.890965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1b560-55be-4bd4-8c9f-003fd324c6f6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.911937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62dab6d-c049-496b-a46b-570be4b4d474 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:42.923638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3023d7-957c-4902-ad32-c880ea487b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.924317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3023d7-957c-4902-ad32-c880ea487b25 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.924600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3023d7-957c-4902-ad32-c880ea487b25 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.924901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3023d7-957c-4902-ad32-c880ea487b25 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.925666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3023d7-957c-4902-ad32-c880ea487b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:42.925965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3023d7-957c-4902-ad32-c880ea487b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:42.927364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3023d7-957c-4902-ad32-c880ea487b25 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:42.939376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 294/565] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:42.941858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d41c9a-5c84-4f7b-a1ce-defb46bd6fae None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.942150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d41c9a-5c84-4f7b-a1ce-defb46bd6fae None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.942454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d41c9a-5c84-4f7b-a1ce-defb46bd6fae None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.944157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 295/566] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:42.944568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d41c9a-5c84-4f7b-a1ce-defb46bd6fae None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:42.947631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:42.948731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:42.949046 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:42.949322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:42.959912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:42.967776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:42.968840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:42.969800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 272/567] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:42.970491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.046137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d41c9a-5c84-4f7b-a1ce-defb46bd6fae None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.066424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e51ca-0203-47bc-a310-3e74fc6fe5b0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.078895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.079562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.079842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.080104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.080867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:43.081180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.082564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.094140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 296/568] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:43.094978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.099481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad443667-5d83-468b-8c10-fb1a14522c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.100316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443667-5d83-468b-8c10-fb1a14522c43 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.100589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443667-5d83-468b-8c10-fb1a14522c43 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.100875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad443667-5d83-468b-8c10-fb1a14522c43 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.108866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad443667-5d83-468b-8c10-fb1a14522c43 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.110316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad443667-5d83-468b-8c10-fb1a14522c43 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ad443667-5d83-468b-8c10-fb1a14522c43 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:43.124431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:43.125515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:43.126454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 273/569] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:43.127033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66614f-7e50-44cd-9b40-287a0cd8eb99 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.128834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66614f-7e50-44cd-9b40-287a0cd8eb99 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.128834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66614f-7e50-44cd-9b40-287a0cd8eb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.145497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-8c66614f-7e50-44cd-9b40-287a0cd8eb99 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:43.146699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8c66614f-7e50-44cd-9b40-287a0cd8eb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:43.194465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5158e-3c2d-4fff-b305-d59a7f675a2f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.214447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 274/570] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:43.215299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66614f-7e50-44cd-9b40-287a0cd8eb99 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.227712 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.228543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.228818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.229113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.236484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.237777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:43.252344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:43.253582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 297/571] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:43.254941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.323434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66614f-7e50-44cd-9b40-287a0cd8eb99 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.354839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84175be0-7852-44f9-8f32-75f89f9a3f76 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.360195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.360878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.361186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.361455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.362225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:43.362505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.363889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.372189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 275/572] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:43.373034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.389830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.390631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.390890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.391161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.398900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.400456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:43.416738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:43.417961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 298/573] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:43.419058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.474734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa99110-e75e-417a-a543-65742e1d60d7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.508613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.509357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.509632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.509897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.510743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:43.511034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.512463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.516703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899913be-a49e-4f8a-b601-69bb40ff3593 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.548553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 276/574] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:43.549465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.552962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.553804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.554087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.554361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.565678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:43.573587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:43.574725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 299/575] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:43.575842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.649054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34fb37-407d-4dfe-82cc-dd633f94c0b5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.675186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc93870-f4ed-4760-8256-f910f8628b4d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.684118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.684782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.685103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.685383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.686181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:43.686486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.687905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.699747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 277/576] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:43.700580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.713572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.714372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.714649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.714917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.725594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.727091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:43.744113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:43.745116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:43.746018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:43.746018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:43.746018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:43.746018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:43.746018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 300/577] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:43.746301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.804992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccee4b-75d7-4a9b-b8f9-b62b471d0aed None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.839004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.839651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.839935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.840218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.841102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:43.841504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.842867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.848211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdcdba-018b-4e32-af00-f37dcd88a119 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:43.862142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 278/578] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:43.863094 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.885626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:43.886452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:43.886730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:43.887005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:43.899547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:43.901084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:43.919563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:43.920596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:43.921802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:43.921802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:43.921802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:43.921802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:43.921802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 301/579] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:43.921802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:43.975814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a875949-d6f4-4ade-8d4e-9590b4699b7b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.014334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.015016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.015273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.015584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.016354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:44.016677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.018177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.027355 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 279/580] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:44.028187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.030793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ec1fd-a0f0-4a0d-99a0-bf8ddacff778 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.070414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.071120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.071476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.071774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.072670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=d4ed1c88f3cd478aafc83f4b76d67f7a) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:44.073016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.074639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.081054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 302/581] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:43 2022] GET /identity/v3/role_assignments?user.id=d4ed1c88f3cd478aafc83f4b76d67f7a&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:44.081885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.159624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03370154-743a-436c-a2c7-01bbddbc137e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.207658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.208415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.208708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.209067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.209787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:44.210099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.211547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.219968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6b602-265d-4dc5-98f9-19d1223691d6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.226111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 280/582] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:44.227185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.264097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.265495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.266050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.266559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.280244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.283366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:44.314515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:44.315648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:44.316689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:44.316689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:44.316689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:44.316689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:44.324004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 303/583] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:44.325672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.364284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb19303-9999-4250-b0d5-00103356be4a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.404291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.405033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.405327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.405720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.406601 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:44.406914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.408457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.426512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 281/584] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:44.426512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.456652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c4e4e-9e45-4e47-88c0-68db0e34952c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.497425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.498273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.498677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.498953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.511801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.513624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:44.534548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:44.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:44.542578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:44.542578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:44.542578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:44.542578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:44.542578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 304/585] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:44.542578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.543020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21459e13-7848-4372-8842-e761dc3579ff None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.580957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.581876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.582160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.582441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.591581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.593107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:44.610657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:44.611763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:44.612866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 282/586] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:44.613507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.692908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82f1c0-8644-4e47-b55b-0b7ffee5e31f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.744576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315214-132b-4709-8391-988411fef577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.745353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315214-132b-4709-8391-988411fef577 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.745625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315214-132b-4709-8391-988411fef577 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.745823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315214-132b-4709-8391-988411fef577 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.745921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94e555-9037-4c3a-b15c-76694f20df45 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.746700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315214-132b-4709-8391-988411fef577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:44.747014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315214-132b-4709-8391-988411fef577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.748481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315214-132b-4709-8391-988411fef577 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.763637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 305/587] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:44.764720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315214-132b-4709-8391-988411fef577 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.782976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.784119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.784549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.785022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.794184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.796589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:44.820642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:44.821788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:44.822795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:44.823328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 283/588] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:44.824283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.901470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315214-132b-4709-8391-988411fef577 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:44.939597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:44.940353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:44.940643 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:44.940949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:44.941689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:44.942001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:44.943450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:44.955855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 306/589] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:44.956824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:44.982404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922c378-f52e-4ea5-ad0d-6599fcbe29ae None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.034122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.034949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.035261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.035612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.036738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:45.037231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.039198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.058607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 284/590] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:45.059530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.083186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641ba29-5f00-44fd-8100-ccbcd0066506 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.133687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.134743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.135055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.135329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.143520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.145151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:45.163887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:45.165357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 307/591] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:45.166391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.188614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d01899-6edb-4cce-b97d-5dd8bd8e122d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.242249 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.243811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.253609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.254112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.255442 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:45.255967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.258011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.277260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 285/592] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:45.278968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.310782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14937323-61de-4780-bd2f-beb5bcf38084 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.352719 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.354097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.354396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.354693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.363682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.365349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:45.392516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:45.394230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 04 14:38:45.395872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:45.401497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 308/593] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:45.402187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.421676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4bd3-215f-4553-8b2f-63f26b6ca92b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.467967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.469023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.469342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.469873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.484165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.486444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:45.508431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:45.509643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:45.516784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:45.516784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:45.516784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:45.516784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:45.516784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 286/594] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:45.516784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.540673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2bdaa-3d65-4a3a-9694-0c5b2ef9866f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.582540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.583480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.583605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.583874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.584678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:45.585024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.586659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.607155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 309/595] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:45.607943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.652884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fef6a-df4a-4c3a-adc7-0569f5807352 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.696765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.697879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.698266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.698646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/fd3eb9e77dc64869b03b1190cf30e54b/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.719375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=fd3eb9e77dc64869b03b1190cf30e54b, role_id=ac8e88511fc54ccda9cfa0f2e06cdaa3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.720906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.737163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 287/596] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:38:45 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/fd3eb9e77dc64869b03b1190cf30e54b/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:45.738577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.750824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406d9d-cecb-42d2-bdbb-94afa77da9ba None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.803972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.804695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.805046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.805372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.806743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:45.807125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.809110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.841112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 310/597] 10.222.0.32 () {56 vars in 1139 bytes} [Wed May 4 14:38:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:45.842260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.902483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfe9bb-4dfc-43fe-98e3-da8192fb36e0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:45.952661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:45.953846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:45.954316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:45.954789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:45.956116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:45.956670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:45.959937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:45.979686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 288/598] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:45.980663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:45.982320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65451686-c663-4a21-939b-f26b7e0b1c64 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.030099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.032460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.032969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.033450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/753f2914e5bc4e298b99529e230d9ee3/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.054572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=753f2914e5bc4e298b99529e230d9ee3, role_id=ac8e88511fc54ccda9cfa0f2e06cdaa3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.056421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.081672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 311/599] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:38:45 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/753f2914e5bc4e298b99529e230d9ee3/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:46.082970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:46.144763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0122cf-94bb-4b87-a5ac-6f9b137f9450 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.207690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.209964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.210541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.211058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.227372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.229965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:46.253431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:46.254434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:46.255435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:46.255435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:46.255435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:46.255435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:46.258955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 289/600] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:46.266204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:46.282269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eab729-973a-4f83-a407-720330710bb8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.368297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.369336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.369614 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.369882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.380386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.381929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:46.403104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:46.404078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:46.404985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 312/601] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:46.413106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:46.461636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0f73-8b15-4430-85b4-831e2d74cd56 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.502470 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5a886-9359-4baf-8409-94929953d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.503583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5a886-9359-4baf-8409-94929953d059 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.504044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5a886-9359-4baf-8409-94929953d059 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.504493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5a886-9359-4baf-8409-94929953d059 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.505480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5a886-9359-4baf-8409-94929953d059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:46.506086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5a886-9359-4baf-8409-94929953d059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.507775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5a886-9359-4baf-8409-94929953d059 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.524031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 290/602] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:46.540639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bd7aa-198a-45bd-89d9-1dd3e782d8d0 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.540993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bd7aa-198a-45bd-89d9-1dd3e782d8d0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.541410 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bd7aa-198a-45bd-89d9-1dd3e782d8d0 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.543189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 291/603] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:46.544003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bd7aa-198a-45bd-89d9-1dd3e782d8d0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:46.553480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682617f9-af53-4faf-8c14-c908b68dde6d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.615662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a9996-7317-44ef-af35-76052461fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.616298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a9996-7317-44ef-af35-76052461fe35 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.616578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a9996-7317-44ef-af35-76052461fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.617023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a9996-7317-44ef-af35-76052461fe35 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.617903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a9996-7317-44ef-af35-76052461fe35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:46.618255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a9996-7317-44ef-af35-76052461fe35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.619740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a9996-7317-44ef-af35-76052461fe35 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.634512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 313/604] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:46.640668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354d360-f4a0-468a-80e9-7338f5febd76 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.642162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354d360-f4a0-468a-80e9-7338f5febd76 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.642445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354d360-f4a0-468a-80e9-7338f5febd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.662430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6354d360-f4a0-468a-80e9-7338f5febd76 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:46.662932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bd7aa-198a-45bd-89d9-1dd3e782d8d0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.663655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6354d360-f4a0-468a-80e9-7338f5febd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:46.700293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.701146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.701430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.701705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.709907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.711391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:46.726475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:46.727564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 292/605] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:46.729566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:46.740529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 314/606] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:46.741541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354d360-f4a0-468a-80e9-7338f5febd76 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:46.844478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354d360-f4a0-468a-80e9-7338f5febd76 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.862846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2048c-55ce-4505-a91e-e27758ca41ed None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:46.880178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.881249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.881537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.881814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.882654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:46.882962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.885138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.900624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:46.905159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:46.905159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:46.905159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/a4aed1720aea4ffa975f7936a1bdb3c1/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:46.905835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 315/607] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:46.905835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:46.922299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=a4aed1720aea4ffa975f7936a1bdb3c1, role_id=ac8e88511fc54ccda9cfa0f2e06cdaa3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:46.923900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:46.948826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 293/608] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:38:46 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/a4aed1720aea4ffa975f7936a1bdb3c1/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:46.948826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:47.058885 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32dd7cd-4c99-4718-ba3b-c4dc2b23e048 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:47.074206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4da68-3b1f-475b-8ebd-1170760a1361 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:47.094898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:47.095795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:47.096080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:47.096356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/0d678d5ee7d74896bbc1316b32ec81ee/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:47.111008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:47.112284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:47.112752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=0d678d5ee7d74896bbc1316b32ec81ee, role_id=ac8e88511fc54ccda9cfa0f2e06cdaa3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:47.113273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:47.113774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:47.114688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:47.129846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:47.140366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:47.141591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:47.142564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:47.146304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 316/609] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:47.156935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 294/610] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:38:46 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/0d678d5ee7d74896bbc1316b32ec81ee/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:47.157225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:47.326249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfbf55-5d6e-4006-90fa-3c8e550ddecf None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:47.370482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:47.370482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:47.370482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:47.370482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:47.371165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:47.371165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:47.372117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:47.383086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 317/611] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:47.390064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:47.523867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8d96a-c303-4147-b764-8a5a0cdd76f8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:47.562447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:47.563666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:47.564177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:47.564648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:47.577117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:47.579004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:47.599193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:47.600168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:47.601101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:47.601101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:47.601101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:47.601101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:47.603581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 295/612] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:47.613978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:47.792946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a9507-10ae-425a-bc61-a1c120385367 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:47.839704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:47.840384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:47.840670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:47.840981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:47.841737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:47.842045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:47.843452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:47.860669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 318/613] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:47.866178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:48.038091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e2d58-a293-48fb-9204-9524fc2e2d35 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:48.073776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:48.074518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:48.074797 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:48.075072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:48.083161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:48.084661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:48.104362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:48.106675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:48.108776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 296/614] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:48.112847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:48.252288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e14422-cf4e-4d7f-956b-681040f407be None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:48.292419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:48.293157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:48.293441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:48.293749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:48.294580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:48.294890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:48.296341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:48.315926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 319/615] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 4 14:38:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:48.328339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:48.459774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983d68f-1bb0-4bff-8ea7-373fd7f36af7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:48.497325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:48.498158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:48.498440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:48.498717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:48.511068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:48.512576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:48.533620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:48.535658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:48.537035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:48.537035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:48.537035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:48.537035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:48.537035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 297/616] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:48.540608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:48.645944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b311d41-247c-4f64-82d6-22b41e154b44 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:48.683408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1db52-4fa3-4212-82fb-844163430b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:48.684114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1db52-4fa3-4212-82fb-844163430b88 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:48.684391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1db52-4fa3-4212-82fb-844163430b88 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:48.684668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1db52-4fa3-4212-82fb-844163430b88 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:48.685447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1db52-4fa3-4212-82fb-844163430b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:48.685753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1db52-4fa3-4212-82fb-844163430b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:48.687175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1db52-4fa3-4212-82fb-844163430b88 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:48.701001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 320/617] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:48.706528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:48.726428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caafcc7-3471-4c3c-9d81-8d414c42e2c7 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:48.726428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caafcc7-3471-4c3c-9d81-8d414c42e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:48.726428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caafcc7-3471-4c3c-9d81-8d414c42e2c7 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:48.726428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 321/618] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:48.734601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e050f-3fa1-4dc7-a1a5-6d74dc921e26 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:48.734601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e050f-3fa1-4dc7-a1a5-6d74dc921e26 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:48.734601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e050f-3fa1-4dc7-a1a5-6d74dc921e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:48.751558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-cb7e050f-3fa1-4dc7-a1a5-6d74dc921e26 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:48.753069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cb7e050f-3fa1-4dc7-a1a5-6d74dc921e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:48.838755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 322/619] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:48.869521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e050f-3fa1-4dc7-a1a5-6d74dc921e26 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:48.917950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781faed-735a-499d-ae3b-b3de62d8f30d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:48.997653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e050f-3fa1-4dc7-a1a5-6d74dc921e26 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:49.026898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.027820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.028112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.028392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.040870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:49.042516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:49.047480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.048328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.048606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.048912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:49.064192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:49.065227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:49.066124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 298/620] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:49.071991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:49.072299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:49.080366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:49.082641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:49.084639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 323/621] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:49.092434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:49.187543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ecf31-9ab3-428d-94b2-43ca991bdff7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:49.224954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.225734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.226021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.226341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.227169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:49.227528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:49.229050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:49.243376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 299/622] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:49.249314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:49.324620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0998b-944c-4ea7-9d0d-61dd6eab44ae None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:49.364079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a871999-534a-4406-aaef-33393e8dc379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.364771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a871999-534a-4406-aaef-33393e8dc379 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.365118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a871999-534a-4406-aaef-33393e8dc379 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.365420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a871999-534a-4406-aaef-33393e8dc379 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.366216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a871999-534a-4406-aaef-33393e8dc379 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:49.366524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a871999-534a-4406-aaef-33393e8dc379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:49.367965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a871999-534a-4406-aaef-33393e8dc379 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:49.375468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8017411-7eeb-454b-b028-e9eb3564c93b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:49.377382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 324/623] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:49.379131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea4d5-3db9-43d2-bfc1-410fedef9584 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.379131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea4d5-3db9-43d2-bfc1-410fedef9584 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.379348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ea4d5-3db9-43d2-bfc1-410fedef9584 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.380742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 325/624] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:49.385772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ea4d5-3db9-43d2-bfc1-410fedef9584 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:49.419250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b92db-bbda-4ab2-b79b-e077ee4c42e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.421140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b92db-bbda-4ab2-b79b-e077ee4c42e0 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.421683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b92db-bbda-4ab2-b79b-e077ee4c42e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.422154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b92db-bbda-4ab2-b79b-e077ee4c42e0 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/d4ed1c88f3cd478aafc83f4b76d67f7a/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.441623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b92db-bbda-4ab2-b79b-e077ee4c42e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=d4ed1c88f3cd478aafc83f4b76d67f7a, role_id=ac8e88511fc54ccda9cfa0f2e06cdaa3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:49.443360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b92db-bbda-4ab2-b79b-e077ee4c42e0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:49.461034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 300/625] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:38:49 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/d4ed1c88f3cd478aafc83f4b76d67f7a/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:38:49.466999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6229720-1f46-4090-af05-8f4133636d15 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.466999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6229720-1f46-4090-af05-8f4133636d15 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.467252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6229720-1f46-4090-af05-8f4133636d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.502646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d6229720-1f46-4090-af05-8f4133636d15 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:49.503735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d6229720-1f46-4090-af05-8f4133636d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:49.531546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ea4d5-3db9-43d2-bfc1-410fedef9584 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:49.611162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.612071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.612356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.612630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.638023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:49.641517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:49.653650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 301/626] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:49.675811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6229720-1f46-4090-af05-8f4133636d15 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:49.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:49.679464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:49.680366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:49.680366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:49.680366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:49.680366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:49.685336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 326/627] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:49.691271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:49.787299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6229720-1f46-4090-af05-8f4133636d15 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:49.822598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f554db0-6bc8-482f-9ab8-3f4bc95d7393 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:49.828916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.829741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.830116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.830400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.845748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:49.859215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:49.860241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:49.861122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 302/628] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:49.866118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:49.867036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:49.867413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:49.867792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:49.868592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:49.868831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:49.869243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:49.870818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:49.886221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 327/629] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:49.894248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.003028 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388e217-6a37-4027-af36-8d28874ead4b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.019315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d63f10-c8eb-48e0-a841-56294a2b3ba8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.038397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63705675-55fa-46b5-90b4-eb0a9b7a004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.038992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63705675-55fa-46b5-90b4-eb0a9b7a004f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.039272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63705675-55fa-46b5-90b4-eb0a9b7a004f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.039546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63705675-55fa-46b5-90b4-eb0a9b7a004f None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.040426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63705675-55fa-46b5-90b4-eb0a9b7a004f None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:50.040644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63705675-55fa-46b5-90b4-eb0a9b7a004f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:50.042117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63705675-55fa-46b5-90b4-eb0a9b7a004f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:50.051167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 303/630] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:50.052514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58200223-e92b-482c-a612-649bb995e919 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.052819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58200223-e92b-482c-a612-649bb995e919 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.057333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58200223-e92b-482c-a612-649bb995e919 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.060097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 304/631] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:50.061527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58200223-e92b-482c-a612-649bb995e919 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.065564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8b82d-529d-482f-84bb-ca76c8362a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.066429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b82d-529d-482f-84bb-ca76c8362a02 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.066706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b82d-529d-482f-84bb-ca76c8362a02 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.066982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8b82d-529d-482f-84bb-ca76c8362a02 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.077416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8b82d-529d-482f-84bb-ca76c8362a02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:50.078904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8b82d-529d-482f-84bb-ca76c8362a02 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-94d8b82d-529d-482f-84bb-ca76c8362a02 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:50.094807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:50.095872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:50.096853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 328/632] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:50.097482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dd359-c836-42ab-8236-c0d5c24eff4f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.098096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dd359-c836-42ab-8236-c0d5c24eff4f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.098096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454dd359-c836-42ab-8236-c0d5c24eff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.114959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-454dd359-c836-42ab-8236-c0d5c24eff4f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:50.116201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-454dd359-c836-42ab-8236-c0d5c24eff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:50.185801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58200223-e92b-482c-a612-649bb995e919 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.188834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 329/633] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:50.189890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87da91-d7cb-4eac-8150-d1c9e1cae2c3 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.190181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87da91-d7cb-4eac-8150-d1c9e1cae2c3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.190460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87da91-d7cb-4eac-8150-d1c9e1cae2c3 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.196152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 330/634] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:50.196152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87da91-d7cb-4eac-8150-d1c9e1cae2c3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.248905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356fde1-ce3f-40a1-97f6-17bfb59047ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.250478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356fde1-ce3f-40a1-97f6-17bfb59047ff None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.250478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356fde1-ce3f-40a1-97f6-17bfb59047ff None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.250833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356fde1-ce3f-40a1-97f6-17bfb59047ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.266083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356fde1-ce3f-40a1-97f6-17bfb59047ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:50.267747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356fde1-ce3f-40a1-97f6-17bfb59047ff None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-a356fde1-ce3f-40a1-97f6-17bfb59047ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:50.286843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:50.288119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 305/635] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b585e98-ca55-4619-9cda-08c50f10a1c0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b585e98-ca55-4619-9cda-08c50f10a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.294832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b585e98-ca55-4619-9cda-08c50f10a1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.311908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-9b585e98-ca55-4619-9cda-08c50f10a1c0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:50.313504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9b585e98-ca55-4619-9cda-08c50f10a1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:50.331897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87da91-d7cb-4eac-8150-d1c9e1cae2c3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.372784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.373494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.373792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.374052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.374916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:50.375227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:50.376695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:50.402200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 331/636] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:50.403036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.411954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 306/637] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:50.413020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b585e98-ca55-4619-9cda-08c50f10a1c0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.521036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691423ea-4fc9-48e7-82dc-45c65f87b262 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.545382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b585e98-ca55-4619-9cda-08c50f10a1c0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.560543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.561394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.561670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.561975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.574462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:50.582611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:50.583824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:50.584769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:50.589208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 332/638] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:50.590135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.598270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.599321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.599605 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.599912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.601003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:50.601322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:50.603209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:50.622274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 307/639] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:50.622274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.711652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a244663-2d46-4cfb-8ab2-7d081564094f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.733197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28c521-e1f8-4398-8bd2-d13693a2a329 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.747285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.748016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.748294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.748577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.770904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.772126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.773114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.773401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.787445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:50.797877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:50.798995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:50.800062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 308/640] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:50.802354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.838593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:50.841602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:50.941909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d2e71-c303-45f9-89b2-87f74b5d72d0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:50.964365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 333/641] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:50.965136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:50.983962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:50.984683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:50.984967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:50.985245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:50.986026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:50.986345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:50.987841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.002374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 309/642] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:51.002374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.086262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff1c83-2e5a-4c94-83f2-678c8985f28c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.123055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7070df-fc29-4c80-8334-668af1b0cc8b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.131040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.132370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.132645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.132958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.144229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.146425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.161057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.161878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.162151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.162453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.163346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:51.163726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.165428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:51.165542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:51.168548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 334/643] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:51.174317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.180057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 310/644] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:51.185672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.294601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf482b06-6faa-4889-b2ae-2ce3b0a843d7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.306552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87506afb-23b3-4132-a630-7fae26b6de83 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.343471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.345767 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.346129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.346427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.362518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.362832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.363519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.363844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.364178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.364278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.376925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.378428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:51.388143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:51.389520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:51.392423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:51.392423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:51.392423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:51.392423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:51.392423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 335/645] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:51.392423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:51.405824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:51.406949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:51.408142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:51.408142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:51.408142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:51.408142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:51.409522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 311/646] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:51.409910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.523838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9829b-90dd-4595-8457-058dd5b46d18 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.547258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c32919-d93f-40c9-96f5-ad89f30a2853 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.563223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.563991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.564374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.564824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.566082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:51.566407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.568356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.592490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 336/647] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:51.593218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.600273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.601130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.601627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.602032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.616067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.617986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:51.637045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:51.638106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:51.639398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:51.639398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:51.639398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:51.639398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:51.643347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 312/648] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:51.644225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.719586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef187559-0604-497e-98b7-43b42d6245f0 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.769135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.769953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.770577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.770858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.771655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:51.772096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.773924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.786565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 337/649] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:51.787436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.788213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814faf-0283-4147-ab65-0bfcbdd11ab1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.835721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.836529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.836813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.837222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.838070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:51.838392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:51.840051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:51.856250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 313/650] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:51.857419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:51.914644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a956662-3a98-46b6-8360-b0c2cbc695ce None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:51.957483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:51.958148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:51.958426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:51.958697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:51.975895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a213b1-6b34-43e1-a7b2-6cb1b9318fac None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.012243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.013118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.013404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.013669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.014389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:52.014688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.016145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.028845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 314/651] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:52.034540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.044339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.045906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.137334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16527a3-709f-47c0-83dd-a6c65cfd0fd4 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.168279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 338/652] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:52.169180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.176924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.177742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.178018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.178289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.185953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.187496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:52.202606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:52.203777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:52.204665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:52.210136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 315/653] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:52.210136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.273603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b10ce-3b43-4fe9-86fb-4baf83e20026 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.310864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d24f2e-66fc-4cdc-bc2e-75761d1941c1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.313709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743948ee-7073-45a8-9f4e-aae9fcaf16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.314632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743948ee-7073-45a8-9f4e-aae9fcaf16b8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.314909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743948ee-7073-45a8-9f4e-aae9fcaf16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.315185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743948ee-7073-45a8-9f4e-aae9fcaf16b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.325257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743948ee-7073-45a8-9f4e-aae9fcaf16b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.326989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743948ee-7073-45a8-9f4e-aae9fcaf16b8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-743948ee-7073-45a8-9f4e-aae9fcaf16b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:52.343066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:52.344321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:52.345745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 339/654] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:52.346499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3953d8-ae2d-41e5-9040-7daeafb9b97d None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.346499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3953d8-ae2d-41e5-9040-7daeafb9b97d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.346646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3953d8-ae2d-41e5-9040-7daeafb9b97d None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.348171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 340/655] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:52.349002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3953d8-ae2d-41e5-9040-7daeafb9b97d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.363361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.364211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.364514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.364837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.373427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.375016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:52.390959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:52.392740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:52.394976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 316/656] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:52.398229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.464194 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3953d8-ae2d-41e5-9040-7daeafb9b97d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.511322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab5091-f52d-4800-a4d4-eaddaa9923b6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.512562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.513456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.513739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.513989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.526425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.528314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:52.548494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:52.549743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:52.550918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:52.550918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:52.550918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:52.550918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:52.550918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 341/657] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:52.550918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.556286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d356d1e-11b7-4109-9fbf-44048743be48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.557054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d356d1e-11b7-4109-9fbf-44048743be48 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.557356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d356d1e-11b7-4109-9fbf-44048743be48 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.557653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d356d1e-11b7-4109-9fbf-44048743be48 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.558525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d356d1e-11b7-4109-9fbf-44048743be48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:52.558855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d356d1e-11b7-4109-9fbf-44048743be48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.560383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d356d1e-11b7-4109-9fbf-44048743be48 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.573437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 317/658] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:52.574898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16d689-4841-4260-97ad-90b91327a41f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.575195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16d689-4841-4260-97ad-90b91327a41f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.575484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16d689-4841-4260-97ad-90b91327a41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.593090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4a16d689-4841-4260-97ad-90b91327a41f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:52.594411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4a16d689-4841-4260-97ad-90b91327a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:52.665000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42b529-4de5-4108-9586-1cba3e16c5bf None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.673277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 318/659] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:52.674155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16d689-4841-4260-97ad-90b91327a41f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.707198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.707905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.708184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.708455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.709378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:52.709693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.711229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.731305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 342/660] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:52.732234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.810812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16d689-4841-4260-97ad-90b91327a41f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.868091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.869200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.869666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.870091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.871236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:52.871738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.874330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.890406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cf8d7-db46-4dc9-9b17-a54a79e413bb None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:52.905241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 319/661] 10.222.0.32 () {56 vars in 1139 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:52.906674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:52.939197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:52.939798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:52.940077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:52.940344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:52.941128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:52.941432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:52.942879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:52.955429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 343/662] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:52.957527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.015487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a4daa-684b-4b14-8311-2c4913cba10d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.052104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.053228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.053717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.054202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.055862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=fd3eb9e77dc64869b03b1190cf30e54b) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:53.056574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.058282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.066860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 320/663] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/role_assignments?user.id=fd3eb9e77dc64869b03b1190cf30e54b&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:53.067689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.070881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1d002-ad26-4c26-9d4c-bf55c0e643aa None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.109885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.110755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.111023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.111301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.122998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:53.131016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:53.132056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:53.132953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 344/664] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:53.133717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.189584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a168c8-83d6-4465-aaf6-40b803a17db5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.226197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.226875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.227151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.227466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.241954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dfaad-f54e-410e-91d2-95bc6bd85ab4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.276417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.277134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.277385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.277659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.302431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.303907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.350330 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.351876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.409142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 321/665] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:53.410052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.460046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 345/666] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:53.461354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.512764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056eb34-fe1f-457e-b2be-72303fc57b8c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.546644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.547450 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.547724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.547991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.557466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.558955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.563097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c5ccd-160b-4498-8df3-4711b53f8f20 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:53.573056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:53.574008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:53.574906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:53.577106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 322/667] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:53.578147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.596028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.596712 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.597014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.597293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.598051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:53.598356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.599731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.612231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 346/668] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:53.612231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.675722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341695d-1cfa-490e-ac4a-29f2cd3fb13c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.707421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760eb3c-2050-4714-b5a5-ac5f59e84c19 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.708109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.708951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.709238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.709512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.721270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.722655 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:53.739541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:53.740519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:53.741467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:53.741467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:53.741467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:53.741467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:53.741690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.741690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.741690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.741690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.745906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 323/669] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:53.746544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.754161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.755629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:53.774804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:53.775768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:53.777863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:53.777863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:53.777863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:53.777863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:53.777863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 347/670] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:53.777863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.853818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a4ef5-9720-40cf-b281-6b5d0c0fb9b7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.874572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6c60-e36d-4b8f-802f-00132425ab56 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:53.886596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.887282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.887567 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.887866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.888638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:53.888982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.890415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.903027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 324/671] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:53.904085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:53.908970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:53.909800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:53.910080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:53.910351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:53.921934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:53.924003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:53.944212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:53.945274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:53.948458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:53.948458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:53.948458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:53.948458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:53.948458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 348/672] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:53.948458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.007194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68ece6-3662-42a7-b698-33fde8ce2be0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.040852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.041565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.041808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.042127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.042816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.043118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.044474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.049605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc6eeb-71e4-4dd0-b34e-e8ac0605f439 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.060454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 325/673] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.060454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.083658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.084338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.084612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.084930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.085698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.086001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.087485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.107016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 349/674] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.107016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.172036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc1160-325e-4ab9-8fcc-bda4495cd098 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.205655 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.206353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.206592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.206860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.207705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=753f2914e5bc4e298b99529e230d9ee3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.208010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.208815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0b10d-e345-4de4-9f8c-52a395ab7cc8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.209503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.217966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 326/675] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/role_assignments?user.id=753f2914e5bc4e298b99529e230d9ee3&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.217966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.254866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.256945 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.257537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.258058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.259139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.259686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.262111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.278170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 350/676] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.279079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.347922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b80fd8-8beb-4a85-9924-1cb0848c443d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.400811 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78344a4-9723-48ab-be0b-5b7a8cf11e97 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.402120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.404341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.404945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.405450 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.415299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.417210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:54.436808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:54.437800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 327/677] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:54.445676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.448836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.449056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.449337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.449614 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.459468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.460975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:54.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:54.479656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:54.480755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:54.481234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 351/678] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:54.481634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.553231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e1cf4-3ba3-41ce-b000-ffc4c21a449a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.597862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d44b0b-fd91-440e-8085-cac6c7218285 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.599191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.600169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.600625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.601066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.608942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.610613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:54.625664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:54.626927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 04 14:38:54.628102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:54.630727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 328/679] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:54.631654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.637685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.638387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.638669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.638949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.639768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.640078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.641562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.655900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 352/680] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.660827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.742127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd52f5-6c41-41fa-b2a0-6b2860d1eb40 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.762441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5955e45-0ed5-4f6b-88db-1b4b4116c080 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.777999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.778649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.778935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.779213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.780311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.780311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.781857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.800333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 329/681] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.801461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.804340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.805484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.806019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.806495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.807613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.808140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.809859 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.829339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 353/682] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.830561 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.924993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8dd17-87f1-454b-98e0-866207af6c39 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.932769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240046-0198-4ca3-b790-916c5b121965 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:54.958971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.959688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.959982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.960266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.961163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=a4aed1720aea4ffa975f7936a1bdb3c1) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.961488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.962987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:54.976993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 330/683] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/role_assignments?user.id=a4aed1720aea4ffa975f7936a1bdb3c1&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:54.976993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:54.977890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:54.981394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:54.981729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:54.982007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:54.984864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=0d678d5ee7d74896bbc1316b32ec81ee) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:54.985198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:54.990716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:55.003766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 354/684] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/role_assignments?user.id=0d678d5ee7d74896bbc1316b32ec81ee&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:55.102897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d7c27-3d5f-415a-a992-2a34c7d77c47 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:55.145131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e5228-302c-475f-b918-4f9ad93b7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:55.145896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e5228-302c-475f-b918-4f9ad93b7700 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:55.146183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e5228-302c-475f-b918-4f9ad93b7700 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:55.146461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e5228-302c-475f-b918-4f9ad93b7700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:55.230316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e5228-302c-475f-b918-4f9ad93b7700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:55.231890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e5228-302c-475f-b918-4f9ad93b7700 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:55.350103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 331/685] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:55.355728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:55.477557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350d96-e5b6-41cc-a880-a2bf5d56ca89 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:55.477776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350d96-e5b6-41cc-a880-a2bf5d56ca89 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:55.478040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350d96-e5b6-41cc-a880-a2bf5d56ca89 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:55.479685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 332/686] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:55.481566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148b4d2-6d3b-4643-b7fa-3e740fc0eedf None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:55.485689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417869c-f87e-40c7-90ff-41b460898ab2 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:55.485973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417869c-f87e-40c7-90ff-41b460898ab2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:55.486234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417869c-f87e-40c7-90ff-41b460898ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:55.504213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7417869c-f87e-40c7-90ff-41b460898ab2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:55.505460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7417869c-f87e-40c7-90ff-41b460898ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:55.527029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:55.527852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:55.528156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:55.528382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:55.546110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:55.547632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:55.569581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:55.570555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:55.571534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:55.571534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:55.571534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:55.571534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:55.571534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 355/687] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:55.577119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:55.671602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 333/688] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:55.683136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57656f-759c-4d42-9fe5-19a36feddc85 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:55.717795 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:55.718381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417869c-f87e-40c7-90ff-41b460898ab2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:55.718491 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:55.718777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:55.719055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:55.719823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:55.720136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:55.721596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:55.734602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 356/689] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:55.740174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:55.846185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c344f-a507-4dcf-8ec2-95671d77ec03 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:55.887817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:55.888882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:55.889163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:55.889515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:55.900824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:55.902657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:55.918470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:55.919410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:55.920322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 357/690] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:55.925926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:55.944196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417869c-f87e-40c7-90ff-41b460898ab2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:55.978580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:55.979766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:55.980263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:55.980728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:55.991656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:55.993588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:56.011580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:56.012539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:56.013488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:56.013488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:56.013488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:56.013488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:56.015745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 334/691] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:56.025442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97c0c-81ad-426e-9170-fb0c86ddcdb4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:56.026089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:56.067605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:56.068285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:56.068573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:56.068876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:56.069687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:56.069998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:56.071544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:56.084355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 358/692] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:56.090456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:56.195762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc6836-9474-46d2-9b25-a1bf8188f50c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:56.233171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:56.233854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:56.234194 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:56.234470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:56.235375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=d4ed1c88f3cd478aafc83f4b76d67f7a) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:56.235692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:56.237196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:56.240838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2e433-e1b1-4404-8051-a7bd2e2f3bd1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:56.243989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 359/693] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:38:56 2022] GET /identity/v3/role_assignments?user.id=d4ed1c88f3cd478aafc83f4b76d67f7a&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:38:56.326329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1cc4e-b472-42fb-8d8d-f54551d04137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:56.327021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1cc4e-b472-42fb-8d8d-f54551d04137 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:56.327302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1cc4e-b472-42fb-8d8d-f54551d04137 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:56.327573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1cc4e-b472-42fb-8d8d-f54551d04137 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:56.328330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1cc4e-b472-42fb-8d8d-f54551d04137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:56.328639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1cc4e-b472-42fb-8d8d-f54551d04137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:56.330165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1cc4e-b472-42fb-8d8d-f54551d04137 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:56.351253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 335/694] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:56.357570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:56.511167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f4695-eeab-4d09-aef0-37c8bdba49af None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:56.551341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a8ba3-be7f-4ff3-9741-2539d0fd2a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:56.552089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8ba3-be7f-4ff3-9741-2539d0fd2a89 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:56.552373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8ba3-be7f-4ff3-9741-2539d0fd2a89 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:56.552647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8ba3-be7f-4ff3-9741-2539d0fd2a89 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:56.553414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a8ba3-be7f-4ff3-9741-2539d0fd2a89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:56.554963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a8ba3-be7f-4ff3-9741-2539d0fd2a89 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:56.584538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 360/695] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:38:56 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:38:56.953200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11466744-3d1a-4fd6-a9f2-97c60cac97cf None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:56.953412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11466744-3d1a-4fd6-a9f2-97c60cac97cf None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:56.953776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11466744-3d1a-4fd6-a9f2-97c60cac97cf None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:56.955381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 336/696] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:56.960926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25510cf4-c27d-418b-ba25-08e29004ae4f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:56.961415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25510cf4-c27d-418b-ba25-08e29004ae4f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:56.961746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25510cf4-c27d-418b-ba25-08e29004ae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:56.983080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-25510cf4-c27d-418b-ba25-08e29004ae4f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:56.984287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-25510cf4-c27d-418b-ba25-08e29004ae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:57.053900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 361/697] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:57.094406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11466744-3d1a-4fd6-a9f2-97c60cac97cf None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:57.222589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11466744-3d1a-4fd6-a9f2-97c60cac97cf None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:57.264534 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:57.265376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:57.265663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:57.265938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:57.285677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:57.293344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:57.295479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:57.297512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:57.304837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 337/698] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:57.305059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25510cf4-c27d-418b-ba25-08e29004ae4f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:57.504591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25510cf4-c27d-418b-ba25-08e29004ae4f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:57.606111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:57.607199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:57.607704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:57.608188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:57.609169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:57.609676 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:57.611267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:57.630956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 362/699] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:57.637145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:57.791822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcefd8-fed8-4798-8ebf-7e5b6c00be1b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:57.834333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fe321-c65e-4076-bd00-0f22c0d467db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:57.835523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe321-c65e-4076-bd00-0f22c0d467db None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:57.836000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe321-c65e-4076-bd00-0f22c0d467db None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:57.836493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe321-c65e-4076-bd00-0f22c0d467db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:57.848965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fe321-c65e-4076-bd00-0f22c0d467db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:57.850815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fe321-c65e-4076-bd00-0f22c0d467db None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-c02fe321-c65e-4076-bd00-0f22c0d467db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:57.869098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:57.870162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:57.871066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:57.871066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:57.871066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:57.871066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:57.873587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 338/700] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:57.879288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:57.968609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569b58b-0ef0-42a4-9565-d15ada458e16 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:57.969075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569b58b-0ef0-42a4-9565-d15ada458e16 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:57.969335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569b58b-0ef0-42a4-9565-d15ada458e16 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:57.971123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 339/701] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:38:57.979571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e192ff7-a354-45fe-9343-1783cc243cde None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:57.979571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e192ff7-a354-45fe-9343-1783cc243cde None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:57.979571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e192ff7-a354-45fe-9343-1783cc243cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:57.998088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4e192ff7-a354-45fe-9343-1783cc243cde None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:57.999279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4e192ff7-a354-45fe-9343-1783cc243cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:58.027193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813dbad-9c71-47d2-9d4c-8ac0f8bd04cc None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.066078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154d02d-1d5a-40cb-b3d9-e90d2f3dd3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.066783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154d02d-1d5a-40cb-b3d9-e90d2f3dd3ae None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.067137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154d02d-1d5a-40cb-b3d9-e90d2f3dd3ae None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.067480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154d02d-1d5a-40cb-b3d9-e90d2f3dd3ae None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.068291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154d02d-1d5a-40cb-b3d9-e90d2f3dd3ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:58.068662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154d02d-1d5a-40cb-b3d9-e90d2f3dd3ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.070169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154d02d-1d5a-40cb-b3d9-e90d2f3dd3ae None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.082919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 363/702] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:58.084369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d8fd-f784-4f48-ba80-122a5708e899 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.084748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d8fd-f784-4f48-ba80-122a5708e899 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.085115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3d8fd-f784-4f48-ba80-122a5708e899 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.086716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 364/703] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:58.089706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3d8fd-f784-4f48-ba80-122a5708e899 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.107988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 340/704] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:58.115459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e951e-b1b5-44ec-a6d4-a8eb9fd5791a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.115459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e951e-b1b5-44ec-a6d4-a8eb9fd5791a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.115459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e951e-b1b5-44ec-a6d4-a8eb9fd5791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.143170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d90e951e-b1b5-44ec-a6d4-a8eb9fd5791a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:58.144376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d90e951e-b1b5-44ec-a6d4-a8eb9fd5791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:58.210547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 341/705] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:58.217456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e951e-b1b5-44ec-a6d4-a8eb9fd5791a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.225411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3d8fd-f784-4f48-ba80-122a5708e899 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.264568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.265771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.266245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.266704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.276519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.278677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:38:58.297601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:58.298621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:38:58.299602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:58.302554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 365/706] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:58.303487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.342653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e951e-b1b5-44ec-a6d4-a8eb9fd5791a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.376357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.377482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.377957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.378435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] PATH_INFO: `/v3/services/network` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.379312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.381000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:58.395205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:58.396326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:38:58.397564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:58.400608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 342/707] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:58.401712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.432879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb3e06-df97-4cd1-993c-b091068227d2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.474626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.475849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.476331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.476892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.477948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.479662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:58.494843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:58.495876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:38:58.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:58.504392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 366/708] 10.222.0.32 () {56 vars in 1129 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:58.505692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.523875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d546785-dfa4-4cc4-b179-6a846c0d2ae1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.558565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.559257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.559538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.559807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.560888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:58.561026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.562455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.580733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 343/709] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:58.581677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.612853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ab985-aae3-4ed2-a0ee-abb9bef641dd None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.647552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.648524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.649114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.649579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.650487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] RBAC: Adding query filter params (name=network) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:58.651021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.652705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.658403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 367/710] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:58.659592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.693018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cbad3-d75e-4ecb-bc46-b6ecb16827de None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.736390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.739147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.739643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.740113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.741055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:58.741715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.744161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.755283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 344/711] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:58.756547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:58.784743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1a175-d9da-4289-8853-6705bc01166a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.826504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.827202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.827479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.827757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.884421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d8586-1429-4368-a566-267d4eb4af36 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:58.898175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.899768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.930358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:58.931991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:58.932763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:58.933270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:58.934200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] RBAC: Adding query filter params (type=network) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:58.934855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:58.937273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:58.945091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 345/712] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:58.946362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.063698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 368/713] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:38:59.069369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.077522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83355871-51a0-4b2e-9261-e999b5a964cf None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.123253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1b0f8-1d3b-4416-944a-97fef5d4d172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.124373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1b0f8-1d3b-4416-944a-97fef5d4d172 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.125527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1b0f8-1d3b-4416-944a-97fef5d4d172 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.126038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1b0f8-1d3b-4416-944a-97fef5d4d172 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.126989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1b0f8-1d3b-4416-944a-97fef5d4d172 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:59.127522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1b0f8-1d3b-4416-944a-97fef5d4d172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:59.129183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1b0f8-1d3b-4416-944a-97fef5d4d172 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:59.135853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 346/714] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 4 14:38:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:59.254665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ed164-7fb7-4aec-95c9-735cfab992cf None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.274370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118fe5-3fea-4d44-b29f-660e73727d4b None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.274720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118fe5-3fea-4d44-b29f-660e73727d4b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.274914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3118fe5-3fea-4d44-b29f-660e73727d4b None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.277901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 347/715] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:59.283731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920f544-63bc-48c4-978f-d671e9014212 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.284062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920f544-63bc-48c4-978f-d671e9014212 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.284290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920f544-63bc-48c4-978f-d671e9014212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.293232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a507c0-6557-452d-bd9b-1282f7165931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.294316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a507c0-6557-452d-bd9b-1282f7165931 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.294743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a507c0-6557-452d-bd9b-1282f7165931 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.295156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a507c0-6557-452d-bd9b-1282f7165931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.303488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0920f544-63bc-48c4-978f-d671e9014212 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:59.305578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0920f544-63bc-48c4-978f-d671e9014212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:59.313311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a507c0-6557-452d-bd9b-1282f7165931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:59.314896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a507c0-6557-452d-bd9b-1282f7165931 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-d7a507c0-6557-452d-bd9b-1282f7165931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:38:59.341931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:59.342925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:59.343819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:38:59.343819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:38:59.343819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:38:59.343819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:59.343819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 369/716] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:38:59.348830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb8337-f6c0-4046-9817-1e9a2a1538c2 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.349682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb8337-f6c0-4046-9817-1e9a2a1538c2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.349974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb8337-f6c0-4046-9817-1e9a2a1538c2 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.351608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 370/717] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:38:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:38:59.352313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb8337-f6c0-4046-9817-1e9a2a1538c2 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.403772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 348/718] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 04 14:38:59.405100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256ced-be3d-4f25-8c0c-459ea1033ea9 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.405353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256ced-be3d-4f25-8c0c-459ea1033ea9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.405630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256ced-be3d-4f25-8c0c-459ea1033ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.429963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-99256ced-be3d-4f25-8c0c-459ea1033ea9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:38:59.431197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-99256ced-be3d-4f25-8c0c-459ea1033ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:38:59.478208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb8337-f6c0-4046-9817-1e9a2a1538c2 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.499406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 349/719] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:38:59.505236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256ced-be3d-4f25-8c0c-459ea1033ea9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.533098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.534690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.535237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.535700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.536756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:59.537273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:59.538932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:59.551851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 371/720] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:59.553227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.611244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256ced-be3d-4f25-8c0c-459ea1033ea9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.645215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.645821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.646099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.646373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.656864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d0500-359a-4f72-9eb8-229b3607aeb3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.657305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:59.665583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:59.666553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:38:59.667390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:59.671682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 350/721] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:59.672549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.696463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.697685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.698176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.698637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.709260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:38:59.717713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:59.718655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:38:59.719570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:38:59.722365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 372/722] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:59.728180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.775922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c826a9-d65e-4cc6-acfa-fb98946f7417 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.823077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.823925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.824211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.824486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.832602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:59.834138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:38:59.849642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:38:59.850615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:38:59.851537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 351/723] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:38:59.852076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.856586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2f08b-f240-4a28-8d33-0ac0b209c7d4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.891025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.892060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.892569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.893091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.894124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:59.894651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:38:59.896227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:38:59.906316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 373/724] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:38:59.907637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:38:59.958712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e71cbd-1938-4ca8-ba13-e67565eb9d2a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:38:59.997558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:38:59.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:38:59.998600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:38:59.998809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:38:59.999593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:38:59.999902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.001426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.015574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 352/725] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:00.016513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.020262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847847e-4ebb-434c-9e9d-363fa0a96e0e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.056236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.057355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.057834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.058334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.059371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:00.059922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.061703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.076127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 374/726] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:38:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:00.077448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.126070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f1bb1-af9c-4203-98ff-a0c104e26a90 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.162871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.163561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.163844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.164118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.179055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.180665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.190611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8716cb-266f-4cc2-84ff-61ff9e39ac94 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:00.199877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:00.200929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:00.201899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:00.201899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:00.201899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:00.201899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:00.205939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 353/727] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:00.206906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.232697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.233998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.234493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.234964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.246887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.248847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:00.268363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:00.269412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:00.270361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:00.270361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:00.270361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:00.270361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:00.273243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 375/728] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:00.274913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.320913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b3ea0-31af-431c-97db-6e7eaf7341e6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.355926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.356616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.356932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.357208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.358078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=753f2914e5bc4e298b99529e230d9ee3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:00.358414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.360303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.366862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 354/729] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/role_assignments?user.id=753f2914e5bc4e298b99529e230d9ee3&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:00.367857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.396009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbe03e-8825-4908-937b-8d53f9f9cb4a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.437278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.438384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.438907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.439371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.440309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:00.440822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.456256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 376/730] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:00.462551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.504504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30604f-fae0-4d5d-9dbe-76c1d6cd1b4d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.540206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.540923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.541206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.541482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.542226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:00.542535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.544056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.571954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199d08b-4f14-49dc-bbec-e19f41061d14 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.575879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 355/731] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:00.581253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.609713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.610905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.611489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.611963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.621223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.623121 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:00.638789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:00.639815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 04 14:39:00.640779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:00.643552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 377/732] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:00.649798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.751511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efed1b6-e392-4f42-9f16-5b9c6e528f97 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.785404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.786465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.786935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.787392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.788420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:00.788966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.790662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.808723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 378/733] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:00.816314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.828937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee5f30-3740-4e1f-ad3c-928d617db780 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:00.887584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:00.888811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:00.889285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:00.889745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:00.903201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:00.905712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:00.928082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:00.929115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:00.930082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:00.941954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 356/734] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:00.951180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:00.998347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab46a3-f7cd-44ed-b32d-650ad8e0633e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:01.060105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:01.061140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.061613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.062095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.094558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d065c24-5118-4224-8b12-da93f864ec09 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:01.150339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:01.151429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.153053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.153531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.154595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:01.155804 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:01.157665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:01.191222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 357/735] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:01.199599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:01.208040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:01.209985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:01.372231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 379/736] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 04 14:39:01.378506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:01.400063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf8d8a-3141-4516-9bd0-9e1255e5afd1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:01.466477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5076e-5ccf-4921-850a-dc277852ec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:01.467156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5076e-5ccf-4921-850a-dc277852ec89 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.467441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5076e-5ccf-4921-850a-dc277852ec89 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.467805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5076e-5ccf-4921-850a-dc277852ec89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.481828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dc283-0975-4205-b496-ff6923f6be70 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:01.550907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c2e9e-21a1-4336-9816-6eb5751321ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:01.551769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c2e9e-21a1-4336-9816-6eb5751321ee None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.552050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c2e9e-21a1-4336-9816-6eb5751321ee None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.552323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c2e9e-21a1-4336-9816-6eb5751321ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.576328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c2e9e-21a1-4336-9816-6eb5751321ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:01.577764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c2e9e-21a1-4336-9816-6eb5751321ee None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:01.595306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5076e-5ccf-4921-850a-dc277852ec89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:01.596830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5076e-5ccf-4921-850a-dc277852ec89 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:01.711809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 358/737] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 04 14:39:01.713258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920d4b-9f9f-4d69-b12a-5855a16128ff None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.713541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920d4b-9f9f-4d69-b12a-5855a16128ff None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.713814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920d4b-9f9f-4d69-b12a-5855a16128ff None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.715266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 359/738] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:01.717694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920d4b-9f9f-4d69-b12a-5855a16128ff None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-0b1c2e9e-21a1-4336-9816-6eb5751321ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:01.744368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:01.745390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:01.746298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:01.746298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:01.746298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:01.746298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:01.746298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 380/739] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:01.747030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7009f-d481-4925-b61d-0eb4f1812200 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.747705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7009f-d481-4925-b61d-0eb4f1812200 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.747705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7009f-d481-4925-b61d-0eb4f1812200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.763975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d0c7009f-d481-4925-b61d-0eb4f1812200 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:01.765245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d0c7009f-d481-4925-b61d-0eb4f1812200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:01.829645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 381/740] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:39:01.831291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f4dbe-06fb-4658-a81d-e102e63a621e None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.831776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920d4b-9f9f-4d69-b12a-5855a16128ff None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:01.832076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f4dbe-06fb-4658-a81d-e102e63a621e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.832542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f4dbe-06fb-4658-a81d-e102e63a621e None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.834027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 382/741] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:01.834922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f4dbe-06fb-4658-a81d-e102e63a621e None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:01.872749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5519c2b-5c77-47bf-b102-bae397ecf814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:01.873593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519c2b-5c77-47bf-b102-bae397ecf814 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.873899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519c2b-5c77-47bf-b102-bae397ecf814 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.874158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519c2b-5c77-47bf-b102-bae397ecf814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.885417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5519c2b-5c77-47bf-b102-bae397ecf814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:01.886876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5519c2b-5c77-47bf-b102-bae397ecf814 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-e5519c2b-5c77-47bf-b102-bae397ecf814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:01.904740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:01.905711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:01.906701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:01.906701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:01.906701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:01.906701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:01.910484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 360/742] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:01.911806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a985ae-cc35-4e91-a7c6-f75c4a6ce0fc None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.912092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a985ae-cc35-4e91-a7c6-f75c4a6ce0fc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.912365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a985ae-cc35-4e91-a7c6-f75c4a6ce0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.928520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a6a985ae-cc35-4e91-a7c6-f75c4a6ce0fc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:01.929745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a6a985ae-cc35-4e91-a7c6-f75c4a6ce0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:01.934470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f4dbe-06fb-4658-a81d-e102e63a621e None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:01.967562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:01.968232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:01.968512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:01.968783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:01.969532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:01.969834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:01.971198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:01.986992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 383/743] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:01.986992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.001061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 361/744] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 04 14:39:02.001061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a985ae-cc35-4e91-a7c6-f75c4a6ce0fc None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.097397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494fc94-df30-4a50-abfa-5cad83102486 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.097691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a985ae-cc35-4e91-a7c6-f75c4a6ce0fc None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.130626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.130935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.131299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.131577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.131676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.131847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.131958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.132171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.132576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.132949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:02.133281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.134018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.134657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.146889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 362/745] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:02.148110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.156548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 384/746] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:01 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:02.157623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.255715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277fa0-159b-4e83-8b2b-11876ce672f5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.278230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753eacfb-5530-4399-9a36-ae9016b71cab None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.292883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.293723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.294009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.294286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.302040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.303517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:02.320401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:02.321644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 363/747] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:02.322572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.338017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.338828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.339171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.339509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.340316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.341876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.361065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 385/748] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:02 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:02.362279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.461349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a86f82-71b3-4140-ae84-7f145c415c01 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.498569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dcd47-1141-4558-8c74-778692ab9f81 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.502421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.503260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.503543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.503821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.512350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.513892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:02.530008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:02.532148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:02.534288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:02.544835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 364/749] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:02.544835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.545342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.545342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.545342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.545342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.545910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:02.545910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.547117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.562621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 386/750] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:02.568276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.662240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b948ce-91f3-4c9e-a255-c93d58c5ae1d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.689323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287fd11-1336-4c06-a815-744cf46e4657 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.726432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.727137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.727471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.727798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.728640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=d4ed1c88f3cd478aafc83f4b76d67f7a) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:02.729019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.730463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.736831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 387/751] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:02 2022] GET /identity/v3/role_assignments?user.id=d4ed1c88f3cd478aafc83f4b76d67f7a&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:02.739336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cdb30-9bec-469a-ba9e-8c46b9935b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.740287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cdb30-9bec-469a-ba9e-8c46b9935b2e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.740770 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cdb30-9bec-469a-ba9e-8c46b9935b2e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.741279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cdb30-9bec-469a-ba9e-8c46b9935b2e None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.742337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cdb30-9bec-469a-ba9e-8c46b9935b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:02.742943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cdb30-9bec-469a-ba9e-8c46b9935b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.744655 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cdb30-9bec-469a-ba9e-8c46b9935b2e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.770896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 365/752] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:02.777810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:02.902768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cda97-74fb-4fc4-80ca-87cddeb3b8f7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:02.938708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4305d0d-bd1e-4ff8-b9bb-61313c45b317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:02.939325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305d0d-bd1e-4ff8-b9bb-61313c45b317 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.939603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305d0d-bd1e-4ff8-b9bb-61313c45b317 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.939873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4305d0d-bd1e-4ff8-b9bb-61313c45b317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.940759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305d0d-bd1e-4ff8-b9bb-61313c45b317 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=b32490f3e59845f0a2720ca879664b21) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:02.941107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305d0d-bd1e-4ff8-b9bb-61313c45b317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:02.942591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4305d0d-bd1e-4ff8-b9bb-61313c45b317 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:02.949377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 388/753] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:02 2022] GET /identity/v3/role_assignments?user.id=b32490f3e59845f0a2720ca879664b21&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:02.952087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21040579-ac23-4175-b3ed-53568796e41c None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.952446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21040579-ac23-4175-b3ed-53568796e41c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.952811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21040579-ac23-4175-b3ed-53568796e41c None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.954502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 366/754] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:02.960635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0dd3-4c38-49ce-9502-cd7e2f88bfae None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:02.960965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0dd3-4c38-49ce-9502-cd7e2f88bfae None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:02.961242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d0dd3-4c38-49ce-9502-cd7e2f88bfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:02.979295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-048d0dd3-4c38-49ce-9502-cd7e2f88bfae None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:02.980553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-048d0dd3-4c38-49ce-9502-cd7e2f88bfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:03.059663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 389/755] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:03.085618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21040579-ac23-4175-b3ed-53568796e41c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:03.215890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21040579-ac23-4175-b3ed-53568796e41c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:03.254246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:03.255551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:03.256030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:03.256498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:03.268140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:03.276699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:03.277696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:03.278563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:03.281377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 367/756] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:03.287637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0dd3-4c38-49ce-9502-cd7e2f88bfae None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:03.390171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d0dd3-4c38-49ce-9502-cd7e2f88bfae None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:03.440506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:03.441239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:03.441521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:03.441796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:03.442576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:03.442883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:03.444302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:03.457441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 390/757] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:03.463092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:03.597272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942fe9d-40d3-4e1b-98a6-3d7ec7891dd3 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:03.655105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:03.655963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:03.656213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:03.656531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:03.673952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:03.675508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:03.697497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:03.698493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:03.699396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:03.699396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:03.699396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:03.699396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:03.699396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 368/758] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:03.705404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:03.828844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fceea-bced-4d07-aef5-9a8110ed8bef None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:03.881156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:03.882521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:03.883045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:03.883645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:03.894272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:03.894272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:03.894272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:03.920688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 391/759] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:03.928119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:04.060327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0c79f-0e2b-4cd5-b4a6-23f0153c2edd None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:04.100398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:04.101308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:04.101589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:04.101868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:04.113896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:04.115355 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:04.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:04.145199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:04.147843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 369/760] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:39:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:04.152918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:04.276881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe3782-7206-440b-ac1c-6f7920580135 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:04.330366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:04.330600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:04.330754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:04.331165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:04.337056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:04.337479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:04.339134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:04.374248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 392/761] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:39:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:04.380648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:04.548342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69524b82-bf34-42c0-8216-2861cadf80b7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:04.592390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:04.593302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:04.593585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:04.593855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:04.609829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:04.611141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:04.648221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:04.650381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:04.652580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:04.652580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:04.652580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:04.652580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:04.652580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 370/762] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:04.656191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:04.820784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b536a2-a425-4445-a658-e9503e5a66f3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:04.865538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:04.866295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:04.866583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:04.866856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:04.867677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:04.867984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:04.869451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:04.895449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 393/763] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:04.901186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:05.006863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a98c8-af18-4c10-a99e-4810dd25abc4 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:05.050221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cdb9b-8c94-4f8c-a66e-d2a0b0339df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:05.051045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cdb9b-8c94-4f8c-a66e-d2a0b0339df4 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.051328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cdb9b-8c94-4f8c-a66e-d2a0b0339df4 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.051591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cdb9b-8c94-4f8c-a66e-d2a0b0339df4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.059588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cdb9b-8c94-4f8c-a66e-d2a0b0339df4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:05.061126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cdb9b-8c94-4f8c-a66e-d2a0b0339df4 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ca8cdb9b-8c94-4f8c-a66e-d2a0b0339df4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:05.080777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:05.081839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:05.082805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 371/764] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:05.088633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:05.218512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875cc3f-b88d-4a30-a995-348fc534269c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:05.266890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:05.267273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e57d-2289-4339-8880-ea56833ea9b3 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.267273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e57d-2289-4339-8880-ea56833ea9b3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.267612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e57d-2289-4339-8880-ea56833ea9b3 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.267770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.267918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.268090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.268933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:05.269267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:05.270384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 372/765] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:05.270716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:05.275203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a303-1aee-46e2-bebc-3132e3d951dc None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.275478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a303-1aee-46e2-bebc-3132e3d951dc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.275769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a303-1aee-46e2-bebc-3132e3d951dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.293821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 394/766] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:05.300703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5f70a303-1aee-46e2-bebc-3132e3d951dc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:05.301013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:05.302045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5f70a303-1aee-46e2-bebc-3132e3d951dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:05.450629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a59865-97a8-498c-8891-70085f4b5c25 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:05.454446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 373/767] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:05.490238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70a303-1aee-46e2-bebc-3132e3d951dc None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:05.492225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b844bd-fb30-426b-a2c9-da1bcf4a4ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:05.493231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b844bd-fb30-426b-a2c9-da1bcf4a4ce7 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.493509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b844bd-fb30-426b-a2c9-da1bcf4a4ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.493782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b844bd-fb30-426b-a2c9-da1bcf4a4ce7 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/753f2914e5bc4e298b99529e230d9ee3/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.498816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b844bd-fb30-426b-a2c9-da1bcf4a4ce7 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=753f2914e5bc4e298b99529e230d9ee3, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:05.500334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b844bd-fb30-426b-a2c9-da1bcf4a4ce7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:05.520407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 395/768] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:39:05 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/753f2914e5bc4e298b99529e230d9ee3/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:39:05.551039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dacba-63ab-486f-8da4-36b545d4cbac None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.551421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dacba-63ab-486f-8da4-36b545d4cbac None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.551586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102dacba-63ab-486f-8da4-36b545d4cbac None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.553150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 396/769] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:05.559981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206689b4-de89-42b7-a015-124cb4056d6f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.560301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206689b4-de89-42b7-a015-124cb4056d6f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.560531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206689b4-de89-42b7-a015-124cb4056d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.587009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-206689b4-de89-42b7-a015-124cb4056d6f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:05.589039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-206689b4-de89-42b7-a015-124cb4056d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:05.690566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70a303-1aee-46e2-bebc-3132e3d951dc None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:05.709926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 397/770] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:05.734714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f3dd2-a3ef-4f0f-8dfb-d4b7d59c5902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:05.735594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f3dd2-a3ef-4f0f-8dfb-d4b7d59c5902 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.735881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f3dd2-a3ef-4f0f-8dfb-d4b7d59c5902 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.736150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f3dd2-a3ef-4f0f-8dfb-d4b7d59c5902 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.736934 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f3dd2-a3ef-4f0f-8dfb-d4b7d59c5902 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:05.738357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f3dd2-a3ef-4f0f-8dfb-d4b7d59c5902 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:05.741379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206689b4-de89-42b7-a015-124cb4056d6f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-4a4f3dd2-a3ef-4f0f-8dfb-d4b7d59c5902 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:05.763485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:05.764563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:05.765436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 374/771] 10.222.0.32 () {56 vars in 1129 bytes} [Wed May 4 14:39:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:05.766834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3272-ef9b-4ecf-a13b-35687c00ac13 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.766948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3272-ef9b-4ecf-a13b-35687c00ac13 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.767135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb3272-ef9b-4ecf-a13b-35687c00ac13 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.772917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 375/772] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:05.772917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb3272-ef9b-4ecf-a13b-35687c00ac13 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:05.874159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206689b4-de89-42b7-a015-124cb4056d6f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:05.885064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb3272-ef9b-4ecf-a13b-35687c00ac13 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:05.919120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d6e09-d79a-433c-a0a8-c51b11918d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:05.919805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:05.920053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d6e09-d79a-433c-a0a8-c51b11918d52 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.920435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d6e09-d79a-433c-a0a8-c51b11918d52 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.920818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d6e09-d79a-433c-a0a8-c51b11918d52 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.921291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.921716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.921835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d6e09-d79a-433c-a0a8-c51b11918d52 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:05.922207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d6e09-d79a-433c-a0a8-c51b11918d52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:05.922353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] PATH_INFO: `/v3/services/network` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.923148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:05.923909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d6e09-d79a-433c-a0a8-c51b11918d52 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:05.924900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:05.929288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 376/773] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 4 14:39:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:05.930961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562655-144a-4a12-a2a7-fbaaca993c0b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:05.931345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562655-144a-4a12-a2a7-fbaaca993c0b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:05.931712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f562655-144a-4a12-a2a7-fbaaca993c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:05.940099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:05.941411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:39:05.942525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:05.945114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 398/774] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:05 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:05.946158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:05.953116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3f562655-144a-4a12-a2a7-fbaaca993c0b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:05.954485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3f562655-144a-4a12-a2a7-fbaaca993c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:06.048881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 377/775] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:06.050139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f562655-144a-4a12-a2a7-fbaaca993c0b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.073061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917c52f-8873-4ac3-b894-4d745a67e767 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.107291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.108346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.108869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.109378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.110342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:06.110887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.112464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.119146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 399/776] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 4 14:39:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:06.120357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.167279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f562655-144a-4a12-a2a7-fbaaca993c0b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.203720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.204606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.204925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.206146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.207025 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] RBAC: Adding query filter params (name=network) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:06.207345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.208762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.216356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 378/777] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:06.217097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.238430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57099012-4766-44bf-b6d5-75c69f80714a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.273537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5830ef3-b94c-4bda-a7a8-4e3346d53204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.274392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5830ef3-b94c-4bda-a7a8-4e3346d53204 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.274675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5830ef3-b94c-4bda-a7a8-4e3346d53204 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.274954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5830ef3-b94c-4bda-a7a8-4e3346d53204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.285831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5830ef3-b94c-4bda-a7a8-4e3346d53204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-e5830ef3-b94c-4bda-a7a8-4e3346d53204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:06.293589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:06.294762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:06.295666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 400/778] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:06.296524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8aa3-f237-4d8c-a30d-2579776a3c7f None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.296827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8aa3-f237-4d8c-a30d-2579776a3c7f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.297113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8aa3-f237-4d8c-a30d-2579776a3c7f None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.298522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 401/779] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:06.301662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a8aa3-f237-4d8c-a30d-2579776a3c7f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.336872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954944e0-dedd-4d74-b57a-51866fa126d9 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.370679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.371349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.371628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.371897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.372692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] RBAC: Adding query filter params (interface=public, service_id=e498cfe8754741d3b5d78e70df7a88e5, region_id=RegionOne) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:06.373047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.374436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.382229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 379/780] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/endpoints?service_id=e498cfe8754741d3b5d78e70df7a88e5&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:06.383087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.418404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a8aa3-f237-4d8c-a30d-2579776a3c7f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.450637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995f72f-6097-438a-b60f-1e1262b01970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.451311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f72f-6097-438a-b60f-1e1262b01970 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.451620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f72f-6097-438a-b60f-1e1262b01970 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.451893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f72f-6097-438a-b60f-1e1262b01970 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.452582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f72f-6097-438a-b60f-1e1262b01970 None admin] RBAC: Adding query filter params (type=network) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:06.452919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f72f-6097-438a-b60f-1e1262b01970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.454323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f72f-6097-438a-b60f-1e1262b01970 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.460284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 402/781] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:06.463723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9baf4-2657-4df3-b048-dc60d2a69563 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.463983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9baf4-2657-4df3-b048-dc60d2a69563 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.465413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9baf4-2657-4df3-b048-dc60d2a69563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.484321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b6b9baf4-2657-4df3-b048-dc60d2a69563 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:06.486883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b6b9baf4-2657-4df3-b048-dc60d2a69563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:06.491479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f357f-ec2f-457c-828d-36973d8654e0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.525274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.525950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.526235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.526506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.527260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:06.527568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.528983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.537705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 380/782] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:06.538419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.568907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 403/783] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:06.568907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9baf4-2657-4df3-b048-dc60d2a69563 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.650459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a31d4-564b-41d0-b207-5f5d94a5ed56 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.662003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9baf4-2657-4df3-b048-dc60d2a69563 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.686049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.686813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.687109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.687376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.688187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.689582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.694925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 381/784] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:06.695782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.707665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.709014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.709758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.710203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.711455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] RBAC: Adding query filter params (interface=public, service_id=abdc65f58663474c957e4a7f0769e692, region_id=RegionOne) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:06.712306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.714182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.726117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 404/785] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/endpoints?service_id=abdc65f58663474c957e4a7f0769e692&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:06.727721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.807370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed390cc-7240-46d9-a99b-f5669b949181 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.842041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d7047-6970-4ec7-bb52-5be01aeae263 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:06.843981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.844877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.845162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.845433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.857493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:06.858972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:06.877555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:06.878586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:06.879515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:06.879515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:06.879515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:06.879515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:06.881716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 382/786] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:06.883172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:06.884749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:06.885626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:06.885905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:06.886176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:06.899494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:06.907304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:06.908309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:06.909188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 405/787] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:06.909889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.016061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51286d-1c90-4ef8-abd8-c2348158c5c5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.036631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac22a5-2c86-46e9-b160-bd0953285d43 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.063007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.063914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.064474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.064754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.065524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.067726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.075499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 383/788] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:07.076375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.090157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.090935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.091216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.091490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.092368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:07.092622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.094129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.128541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 406/789] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:07.129879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.232959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8e33a-32e9-4544-a0df-3fc1c5776a85 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.304915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.306110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.307136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.307629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.308632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:07.310263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.311952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.335486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 384/790] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:07.343472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.363308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5caac-56ff-4941-9970-446b95103085 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.411444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.412816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.413119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.413396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.424652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.426540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:07.443343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:07.444370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 04 14:39:07.445358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:07.448167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 407/791] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:07.454658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.476183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03921462-6ffd-44fc-8ceb-03bdb3802b6e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.513642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.514872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.515354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.515820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.528549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.530423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:07.549390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:07.550368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:07.551269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:07.551269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:07.551269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:07.551269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:07.553628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 385/792] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:07.562989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.569366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e90c02-f030-41d3-af76-0a7640e54d17 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.653091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.653701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.653984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.654258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.655120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:07.655430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.657000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.703355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 408/793] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:07.726024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.743319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4753b4-e1ab-4f8d-807d-65e564069dcb None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.784039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.784742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.787336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.787656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.788460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:07.788775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.790328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.803559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 386/794] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:07.813892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.862848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b86fd-feca-4938-a7ed-d76aa01759a0 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:07.907552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a41e5f-6034-45d9-a917-337bb50d077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:07.909231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a41e5f-6034-45d9-a917-337bb50d077c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.909740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a41e5f-6034-45d9-a917-337bb50d077c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.910213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a41e5f-6034-45d9-a917-337bb50d077c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.927432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a41e5f-6034-45d9-a917-337bb50d077c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:07.930084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a41e5f-6034-45d9-a917-337bb50d077c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-90a41e5f-6034-45d9-a917-337bb50d077c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:07.957731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:07.958743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:07.960347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:07.960347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:07.960347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:07.960347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:07.960347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 409/795] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:07.960653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55541f32-673a-49a5-a711-fdbfc9348322 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:07.960963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55541f32-673a-49a5-a711-fdbfc9348322 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:07.961226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55541f32-673a-49a5-a711-fdbfc9348322 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:07.966606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 410/796] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:07.967479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55541f32-673a-49a5-a711-fdbfc9348322 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:07.987292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d25160-b851-41d2-9b09-fc4eab9540dd None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.028741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6593ed7-ea83-439c-ad02-2c2c10db063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.030923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6593ed7-ea83-439c-ad02-2c2c10db063c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.031514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6593ed7-ea83-439c-ad02-2c2c10db063c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.031926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6593ed7-ea83-439c-ad02-2c2c10db063c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.041971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6593ed7-ea83-439c-ad02-2c2c10db063c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.044371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6593ed7-ea83-439c-ad02-2c2c10db063c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-c6593ed7-ea83-439c-ad02-2c2c10db063c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:08.064917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:08.066185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:08.067460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:08.071170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 387/797] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:08.073574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4a15-d93b-4fff-bb8a-44ca0eb76c4a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.074294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4a15-d93b-4fff-bb8a-44ca0eb76c4a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.074781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4a15-d93b-4fff-bb8a-44ca0eb76c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.083361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55541f32-673a-49a5-a711-fdbfc9348322 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.098110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-83fb4a15-d93b-4fff-bb8a-44ca0eb76c4a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:08.099980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-83fb4a15-d93b-4fff-bb8a-44ca0eb76c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:08.123797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.124659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.125031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.125395 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.126230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:08.126625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.130981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.147654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 411/798] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:08.148877 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.181958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 388/799] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:08.182996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb4a15-d93b-4fff-bb8a-44ca0eb76c4a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.261582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a4a4-61b1-4dd5-a134-3f237e0caa62 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.296198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.297193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.297622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.298047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.299098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:08.299571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.301208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.301913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb4a15-d93b-4fff-bb8a-44ca0eb76c4a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.320349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 412/800] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:08.321712 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.341226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.342237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.342637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.343009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.350346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.352045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:08.367513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:08.368757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:08.369949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:08.371757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 389/801] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:08.372510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.432698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03c443-5cfd-4676-a215-8c5d7170303c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.468485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.469628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.470082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.470497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.479063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6dd54-5e81-4603-b1a8-62975c80c08f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.482801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:08.490769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:08.491831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:08.492760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:08.495644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 413/802] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:08.496773 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.521367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.522566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.522844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.523113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.534616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.536069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:08.553878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:08.554889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:08.556460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:08.556460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:08.556460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:08.556460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:08.556460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 390/803] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:08.556460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.609087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9416d-36bf-44ad-bce4-d0ca3ab5eed1 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.644356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.645457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.645973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.646460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.647840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:08.648351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.650262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.663145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5edc0-1140-4834-ae09-48d49029dae7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.664969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 414/804] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:08.666156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.703952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.704557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.704930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.705206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.706005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:08.706312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.707712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.721469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 391/805] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:08.722393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.775765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec1a6a-fc38-4480-ba67-f8a2563a00e4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.810446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.811508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.812021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.812513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.813492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:08.814004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.815616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.822397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2292888-b959-4ba7-ac92-5cd213a3141f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:08.833475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 415/806] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:08.835377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.857473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:08.858371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:08.858647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:08.859290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/d4ed1c88f3cd478aafc83f4b76d67f7a/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:08.860946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=d4ed1c88f3cd478aafc83f4b76d67f7a, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:08.862498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:08.885656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 392/807] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:39:08 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/d4ed1c88f3cd478aafc83f4b76d67f7a/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:39:08.886767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:08.964047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb98e-4149-4a0d-be30-b68391a9235b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.002258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0d4c9-b25b-451e-9efa-76d2ad5e2ae3 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.009416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.010279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.010556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.010828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.033467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.035763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.064402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.069670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.070268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.070849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:09.077213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:09.078599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:09.079506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:09.079506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:09.079506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:09.079506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:09.086720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 416/808] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:09.088236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.089954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.092009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:09.112956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:09.113950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:09.114879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:09.117763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 393/809] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:09.125940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.210819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3be316-d08b-443e-b77d-57fc47a4e427 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.247589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7898d48-5e82-4274-976a-49e7d10da388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.248205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7898d48-5e82-4274-976a-49e7d10da388 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.248516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7898d48-5e82-4274-976a-49e7d10da388 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.248748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7898d48-5e82-4274-976a-49e7d10da388 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.249658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7898d48-5e82-4274-976a-49e7d10da388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:09.249856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7898d48-5e82-4274-976a-49e7d10da388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.251245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7898d48-5e82-4274-976a-49e7d10da388 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.251993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02301693-b4d3-491a-97f2-e6ba4b8aa970 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.264061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 417/810] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:09.265304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a199fb-9312-47a1-8ee7-07c9a13f7566 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.265587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a199fb-9312-47a1-8ee7-07c9a13f7566 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.265857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a199fb-9312-47a1-8ee7-07c9a13f7566 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.268035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 418/811] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:09.273860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879282-af1a-4c69-9331-71a7b3fdd6a8 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.274026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879282-af1a-4c69-9331-71a7b3fdd6a8 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.274246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76879282-af1a-4c69-9331-71a7b3fdd6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.291233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-76879282-af1a-4c69-9331-71a7b3fdd6a8 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:09.292456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-76879282-af1a-4c69-9331-71a7b3fdd6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:09.294554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.294554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.294554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.294554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.295908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:09.295908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.297164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.310897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 394/812] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:09.311879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.400499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 419/813] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:09.400499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76879282-af1a-4c69-9331-71a7b3fdd6a8 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.438939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d1242-2143-4b28-881a-78dc3731518f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.473937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.474848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.475207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.475556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] PATH_INFO: `/v3/users/nova` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.484207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.485861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:09.500901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:09.501927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:09.502868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:09.507065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 395/814] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:09.507982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.515623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76879282-af1a-4c69-9331-71a7b3fdd6a8 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.549387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.550603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.551057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.551500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/b32490f3e59845f0a2720ca879664b21/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.567292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=b32490f3e59845f0a2720ca879664b21, role_id=ac8e88511fc54ccda9cfa0f2e06cdaa3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.569064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.584868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 420/815] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:39:09 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/b32490f3e59845f0a2720ca879664b21/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:39:09.586706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.614794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bcf8b-3115-4d87-bd95-2cc0caceb34d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.659259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.661092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.661595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.662303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.663234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.665940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:09.682287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:09.683488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 04 14:39:09.684511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:09.688377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 396/816] 10.222.0.32 () {56 vars in 1129 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:09.695570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.717847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995be9e-7c67-41ae-b4ff-2f458ce42358 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.755972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.757024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.757378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.757762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.758797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:09.759336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.761201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.786656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 421/817] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:09.788074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.811268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a493-5193-44a5-a7e4-90eaecc934ea None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.847784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.848897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.849391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.849848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:09.850807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:09.851332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:09.853110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:09.859041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 397/818] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:09.881958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:09.921892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a9f8a-1bd6-4839-9112-f6a496cc6397 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:09.961848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aa8a2-7557-478e-8a29-1d484825718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:09.962639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa8a2-7557-478e-8a29-1d484825718f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:09.962920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa8a2-7557-478e-8a29-1d484825718f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:09.963194 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa8a2-7557-478e-8a29-1d484825718f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.007941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f51a87-247e-4f77-b28d-8f1000534321 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:10.049933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78aa8a2-7557-478e-8a29-1d484825718f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.051805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78aa8a2-7557-478e-8a29-1d484825718f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.074286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a16e09-0606-45c8-907f-3444564275dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:10.075295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a16e09-0606-45c8-907f-3444564275dd None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.075579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a16e09-0606-45c8-907f-3444564275dd None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.075855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a16e09-0606-45c8-907f-3444564275dd None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.076848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a16e09-0606-45c8-907f-3444564275dd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:10.077170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a16e09-0606-45c8-907f-3444564275dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.078590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a16e09-0606-45c8-907f-3444564275dd None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.088929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 398/819] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:10.090486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9566c-ff37-4bc8-8227-6e7e54c46a2b None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.090781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9566c-ff37-4bc8-8227-6e7e54c46a2b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.091113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9566c-ff37-4bc8-8227-6e7e54c46a2b None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.093666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 399/820] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:10.094497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9566c-ff37-4bc8-8227-6e7e54c46a2b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:10.190608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 422/821] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:10.192397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305c6ab-3731-464f-89a4-8fefe5ad1e42 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.192935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305c6ab-3731-464f-89a4-8fefe5ad1e42 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.193409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305c6ab-3731-464f-89a4-8fefe5ad1e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.210039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6305c6ab-3731-464f-89a4-8fefe5ad1e42 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:10.211663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6305c6ab-3731-464f-89a4-8fefe5ad1e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:10.225721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9566c-ff37-4bc8-8227-6e7e54c46a2b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:10.267987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:10.268579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.268900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.269415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.270189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.272414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.282663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 423/822] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:10.283945 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305c6ab-3731-464f-89a4-8fefe5ad1e42 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:10.300889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 400/823] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:10.317430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:10.493856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305c6ab-3731-464f-89a4-8fefe5ad1e42 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:10.537976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58952f83-0606-46f3-98e8-904fe99d5a9e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:10.572721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:10.573562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.575068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.575068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.583287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:10.584861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.585602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.586110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.586393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] PATH_INFO: `/v3/services/network` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.587293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.587574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.589233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:10.605029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:10.606093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 04 14:39:10.606963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:10.609970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 401/824] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:10 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:10.610098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:10.610970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:10.611882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:10.611882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:10.611882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:10.611882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:10.616846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 424/825] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:10.618073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:10.623476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:10.746207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40056672-90bf-4ef0-abda-a70e44341153 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:10.759061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cc5dc-62cc-458c-8ff2-46b6d06f9674 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:10.785195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:10.786233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.786690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.787147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.788156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:10.788659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.790379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.806310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:10.807016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.807203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 425/826] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:10.807310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.807555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.808287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] RBAC: Adding query filter params (name=network) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:10.808597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.810105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.815845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 402/827] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:10.817892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:10.825899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:10.918185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fcc0f-c333-4df0-9058-382806028a71 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:10.954966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:10.955796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:10.956076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:10.956348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:10.971875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:10.973436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:10.998810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:10.999829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:11.001571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 426/828] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:11.007012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:11.048306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9feee-63d6-41ec-963b-60df2d300f83 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:11.113865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:11.114807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:11.115091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:11.116931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:11.117778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:11.118091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:11.119522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:11.133775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 403/829] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:11.139799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:11.171218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33118e4a-54bb-4351-946b-632b12f32a60 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:11.217269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:11.217974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:11.218262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:11.218538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:11.219362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:11.219673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:11.221195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:11.234668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 427/830] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:11.245995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:11.268268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc712dc-1e03-4874-a9ed-d9d7730f91a4 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:11.334586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1451d-ded8-44cf-ab85-6151d8549504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:11.339933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1451d-ded8-44cf-ab85-6151d8549504 None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:11.340248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1451d-ded8-44cf-ab85-6151d8549504 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:11.340476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1451d-ded8-44cf-ab85-6151d8549504 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:11.341293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1451d-ded8-44cf-ab85-6151d8549504 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:11.342700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1451d-ded8-44cf-ab85-6151d8549504 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:11.381449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 404/831] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:11.461418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8a82-de9d-49e3-886e-47905b21a689 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:11.552042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03ef4-e2f3-44bb-89ea-78f60bf06be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:11.552727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03ef4-e2f3-44bb-89ea-78f60bf06be5 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:11.553030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03ef4-e2f3-44bb-89ea-78f60bf06be5 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:11.553304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03ef4-e2f3-44bb-89ea-78f60bf06be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:11.554314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03ef4-e2f3-44bb-89ea-78f60bf06be5 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=753f2914e5bc4e298b99529e230d9ee3) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:11.554622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03ef4-e2f3-44bb-89ea-78f60bf06be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:11.556170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03ef4-e2f3-44bb-89ea-78f60bf06be5 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:11.566473 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 428/832] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:11 2022] GET /identity/v3/role_assignments?user.id=753f2914e5bc4e298b99529e230d9ee3&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:11.781215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947151-b869-45da-a451-5d679cc76d23 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:11.781646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947151-b869-45da-a451-5d679cc76d23 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:11.781819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947151-b869-45da-a451-5d679cc76d23 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:11.783461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 405/833] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:11.789626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6034-2942-4251-8bbc-322cccd8373b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:11.789949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6034-2942-4251-8bbc-322cccd8373b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:11.790149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6034-2942-4251-8bbc-322cccd8373b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:11.811979 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a6fd6034-2942-4251-8bbc-322cccd8373b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:11.813328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a6fd6034-2942-4251-8bbc-322cccd8373b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:11.888354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 429/834] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:11.913288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947151-b869-45da-a451-5d679cc76d23 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.030086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947151-b869-45da-a451-5d679cc76d23 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.064777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef1984-442f-49a4-8b5d-d5921e64e1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.065538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1984-442f-49a4-8b5d-d5921e64e1b2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.065815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1984-442f-49a4-8b5d-d5921e64e1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.066086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef1984-442f-49a4-8b5d-d5921e64e1b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.077155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef1984-442f-49a4-8b5d-d5921e64e1b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-1fef1984-442f-49a4-8b5d-d5921e64e1b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:12.085093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:12.086086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:12.086949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 406/835] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:12.092542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd6034-2942-4251-8bbc-322cccd8373b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.191285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd6034-2942-4251-8bbc-322cccd8373b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.201181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84060b5c-8ac2-4888-a15b-595734cc2c6d None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.201787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84060b5c-8ac2-4888-a15b-595734cc2c6d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.201787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84060b5c-8ac2-4888-a15b-595734cc2c6d None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.203368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 407/836] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:12.208269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727312f6-25ac-4a55-a65e-6dbad763f337 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.208588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727312f6-25ac-4a55-a65e-6dbad763f337 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.208843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727312f6-25ac-4a55-a65e-6dbad763f337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.226593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-727312f6-25ac-4a55-a65e-6dbad763f337 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:12.227865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-727312f6-25ac-4a55-a65e-6dbad763f337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:12.234429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.235427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.236355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.236899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.238244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:12.238775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:12.241003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:12.252722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 430/837] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:12.258502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.321733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 408/838] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:12.345512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727312f6-25ac-4a55-a65e-6dbad763f337 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.390516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a48ce-3ca4-4a38-8d29-0006d20cb194 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.432312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.433709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.433988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.434262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.448934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:12.450391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:12.466346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727312f6-25ac-4a55-a65e-6dbad763f337 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:12.467500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:12.468379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:12.474546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:12.474546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:12.474546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:12.474546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:12.474546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 431/839] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:12.474827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.512300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.513148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.513430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.513703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.525805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:12.533943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:12.534951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:12.535891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 409/840] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:12.541565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.592622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce1dd5-4ee2-49ae-aaa4-255a6e63e111 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.628768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.629863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.630297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.630719 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.631607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:12.632078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:12.633729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:12.652285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a206a5-c7a3-4d22-9856-89b6f42a2ec2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.652635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 432/841] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:12.653941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.689310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.690062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.690339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.690614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.691474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:12.691683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:12.693302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:12.707430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 410/842] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:12.708518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.759979 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d71c6-f261-45e8-9ee8-d8b9fd6bc450 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.794725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.795859 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.796288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.796710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.805072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:12.806901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:12.817589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4f8db-fb2c-43cb-8983-1db25425cee5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:12.823596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:12.824869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 04 14:39:12.825925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:12.832941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 433/843] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:12.834226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.858041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.858776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.859056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.859425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.871742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:12.873355 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:12.893024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:12.894177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:12.895208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:12.895208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:12.895208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:12.895208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:12.899705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 411/844] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:12.900666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:12.943762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc0e47-296a-4791-b57c-35a3a206dc15 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:12.977805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:12.978743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:12.979193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:12.979600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:12.980606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:12.981080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:12.982686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.005123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a08e-e740-43f5-ac85-d7cabb2fa327 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.005459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 434/845] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:13.006632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.035748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2e86a-063c-4f49-9dda-576a991ff3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.036418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e86a-063c-4f49-9dda-576a991ff3a3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.036697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e86a-063c-4f49-9dda-576a991ff3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.036996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2e86a-063c-4f49-9dda-576a991ff3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.037750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2e86a-063c-4f49-9dda-576a991ff3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:13.038055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2e86a-063c-4f49-9dda-576a991ff3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.039540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2e86a-063c-4f49-9dda-576a991ff3a3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.057064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 412/846] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:13.058242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb413d7-af9d-499a-8a4d-827c8ec04046 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.058530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb413d7-af9d-499a-8a4d-827c8ec04046 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.058776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb413d7-af9d-499a-8a4d-827c8ec04046 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.064253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 413/847] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:13.064253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb413d7-af9d-499a-8a4d-827c8ec04046 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.115213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45c2de-4111-475a-aadc-bcee21135065 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.149467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.150120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.150399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.150675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.162825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb413d7-af9d-499a-8a4d-827c8ec04046 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.195499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f6780-20ab-40c2-90b9-23961a3afcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.196315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f6780-20ab-40c2-90b9-23961a3afcfa None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.196612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f6780-20ab-40c2-90b9-23961a3afcfa None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.196897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f6780-20ab-40c2-90b9-23961a3afcfa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.204105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f6780-20ab-40c2-90b9-23961a3afcfa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.205597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f6780-20ab-40c2-90b9-23961a3afcfa None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.215495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.217015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-d54f6780-20ab-40c2-90b9-23961a3afcfa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:13.220406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:13.221421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 414/848] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acea057-afc3-467b-bebc-a10256887a55 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.227402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acea057-afc3-467b-bebc-a10256887a55 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.228119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acea057-afc3-467b-bebc-a10256887a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.243289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-1acea057-afc3-467b-bebc-a10256887a55 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:13.244478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-1acea057-afc3-467b-bebc-a10256887a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:13.307516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 415/849] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:13.308441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acea057-afc3-467b-bebc-a10256887a55 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.338108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 435/850] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 04 14:39:13.339040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.420950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acea057-afc3-467b-bebc-a10256887a55 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.450200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7e1db-f03c-42ca-9e3d-fa107b383d0c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.466683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.467929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.467929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.468116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.468928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:13.469239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.470683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.485281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.486117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.486394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.486667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.487332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.488703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.493798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 416/851] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:13.493798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:13.504623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:13.511170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:13.512101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 436/852] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:13.512956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.595590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44843016-73ae-4b0a-97d6-b1fed294c01a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.617043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc006c1b-2e1a-445f-8db4-7b963e908bb2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.630634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.631456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.631814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.632095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.642731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.644191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.653053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.653715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.653995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.654268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:13.663099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:13.664063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:13.665089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:13.665089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:13.665089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:13.665089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:13.665089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 417/853] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:13.665388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.729188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.730591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.764176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbdf6c-2753-413d-81b4-0ffc7675a496 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.844432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 437/854] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:13.844432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.928271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.928978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.929261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.929531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.930415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:13.930718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.932072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:13.941505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 418/855] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:13.941505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:13.948448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2daddf0-178d-447a-8302-0c82f54a79c2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:13.983388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:13.983975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:13.984241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:13.984535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:13.985272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:13.985601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:13.986982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.002947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 438/856] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:14.002947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.043217 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63186c8-1d28-426a-914b-aa1b3d913bbf None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.074525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.075318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.075593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.075857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.086009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.087453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.098297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f0824-c84d-4275-8cf7-906821c70d35 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:14.104630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:14.105551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:14.106446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:14.106446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:14.106446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:14.106446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:14.106446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 419/857] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:14.106721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.131803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab805522-ebaa-4471-9865-8af5c523cd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.132489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab805522-ebaa-4471-9865-8af5c523cd15 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.132869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab805522-ebaa-4471-9865-8af5c523cd15 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.133174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab805522-ebaa-4471-9865-8af5c523cd15 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.133880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab805522-ebaa-4471-9865-8af5c523cd15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:14.134187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab805522-ebaa-4471-9865-8af5c523cd15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.135562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab805522-ebaa-4471-9865-8af5c523cd15 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.140214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 439/858] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:14.141297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4109d-5b3d-4fc1-8771-2715e1e5271d None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.141585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4109d-5b3d-4fc1-8771-2715e1e5271d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.141862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4109d-5b3d-4fc1-8771-2715e1e5271d None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.143224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 440/859] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:14.143800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4109d-5b3d-4fc1-8771-2715e1e5271d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.221189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19571f-5b6f-47ad-9224-6378f93bac08 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.251425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4109d-5b3d-4fc1-8771-2715e1e5271d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.260750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d88829-a674-460a-9664-7208c02a9623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.261670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d88829-a674-460a-9664-7208c02a9623 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.261950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d88829-a674-460a-9664-7208c02a9623 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.262220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d88829-a674-460a-9664-7208c02a9623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.269453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d88829-a674-460a-9664-7208c02a9623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.270908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d88829-a674-460a-9664-7208c02a9623 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-30d88829-a674-460a-9664-7208c02a9623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:14.286069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:14.287238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:14.288267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 420/860] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:14.289180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53555-7771-4cd3-acce-e47680b94894 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.289462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53555-7771-4cd3-acce-e47680b94894 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.289736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53555-7771-4cd3-acce-e47680b94894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.294202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.294887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.295141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.295418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.296195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:14.296477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.297974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.310739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5fb53555-7771-4cd3-acce-e47680b94894 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:14.311948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5fb53555-7771-4cd3-acce-e47680b94894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:14.315368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 441/861] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:14.316597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.389633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 421/862] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:14.390514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb53555-7771-4cd3-acce-e47680b94894 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.450732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bca60-f61b-43f0-bd10-fcc66bd25883 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.493559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb53555-7771-4cd3-acce-e47680b94894 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.493810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.494363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.494639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.495068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.496724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:14.497054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.498487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.511461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 442/863] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:14.513421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.534852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.535725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.535998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.536747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.543906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.545443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:14.559242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:14.560258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 422/864] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:14.561569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.617671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d25d2-6921-43a3-9764-dfe94fab973e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.651014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.652173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.652633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.653112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.653965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.655530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.668702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edd15c-1b05-4040-9f0a-fe808cd65375 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:14.669294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:14.670242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:14.671281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:14.673608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 443/865] 10.222.0.32 () {56 vars in 1132 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:14.674546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.704966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.705629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.705917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.706195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.707059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=d4ed1c88f3cd478aafc83f4b76d67f7a) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:14.707346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.708769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.714575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 423/866] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/role_assignments?user.id=d4ed1c88f3cd478aafc83f4b76d67f7a&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:14.715373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.794354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334f7df-6016-4884-9aaa-daec21719686 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.829989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c12d71-445f-45bf-9b6f-6f5c4131b721 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:14.841083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.841790 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.842090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.842357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.843745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:14.844061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.846312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.864235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 444/867] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:14.871837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.877365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:14.878504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:14.879004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:14.879506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:14.880434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:14.880994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:14.882638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:14.888504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 424/868] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:14.893915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:14.988476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2132b-6811-45e7-b6f8-8f952ba68383 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:15.007829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4701495-9cee-4447-ba42-e5cab71c81ac None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:15.027310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea1855-6225-407b-910a-81ed8f063f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:15.028038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1855-6225-407b-910a-81ed8f063f9b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:15.028321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1855-6225-407b-910a-81ed8f063f9b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:15.028655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea1855-6225-407b-910a-81ed8f063f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:15.029539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea1855-6225-407b-910a-81ed8f063f9b None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=b32490f3e59845f0a2720ca879664b21) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:15.029850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea1855-6225-407b-910a-81ed8f063f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:15.031347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea1855-6225-407b-910a-81ed8f063f9b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:15.040163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 445/869] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/role_assignments?user.id=b32490f3e59845f0a2720ca879664b21&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:15.049300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bfea6-364b-4814-bc6c-157b9e03ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:15.049915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bfea6-364b-4814-bc6c-157b9e03ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:15.050192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bfea6-364b-4814-bc6c-157b9e03ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:15.050522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bfea6-364b-4814-bc6c-157b9e03ceb6 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:15.051255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bfea6-364b-4814-bc6c-157b9e03ceb6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:15.051603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bfea6-364b-4814-bc6c-157b9e03ceb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:15.053078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bfea6-364b-4814-bc6c-157b9e03ceb6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:15.059122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 425/870] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:16.436209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c4e31-18a3-48cd-b628-d305ba5931a1 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:16.436745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c4e31-18a3-48cd-b628-d305ba5931a1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:16.436745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c4e31-18a3-48cd-b628-d305ba5931a1 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:16.438422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 446/871] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:16.443805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5022562-0220-48b7-af7f-8356aa48de92 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:16.444233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5022562-0220-48b7-af7f-8356aa48de92 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:16.444327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5022562-0220-48b7-af7f-8356aa48de92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:16.461478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a5022562-0220-48b7-af7f-8356aa48de92 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:16.462644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a5022562-0220-48b7-af7f-8356aa48de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:16.529750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 426/872] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:16.552089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c4e31-18a3-48cd-b628-d305ba5931a1 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:16.656692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c4e31-18a3-48cd-b628-d305ba5931a1 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:16.694036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8261c81-2202-45e8-a8e9-4acbb005ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:16.694724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261c81-2202-45e8-a8e9-4acbb005ee92 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:16.695000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261c81-2202-45e8-a8e9-4acbb005ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:16.695306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261c81-2202-45e8-a8e9-4acbb005ee92 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:16.696090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8261c81-2202-45e8-a8e9-4acbb005ee92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:16.697501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8261c81-2202-45e8-a8e9-4acbb005ee92 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:16.713698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 447/873] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:16 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:17.074468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a383fbf-f482-4026-ad2c-b2032ec8b6bc None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.075196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a383fbf-f482-4026-ad2c-b2032ec8b6bc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.075697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a383fbf-f482-4026-ad2c-b2032ec8b6bc None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.077532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 427/874] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:17.083284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab19e9-0342-46e7-9071-61102051d033 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.083560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab19e9-0342-46e7-9071-61102051d033 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.083835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab19e9-0342-46e7-9071-61102051d033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.100880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-58ab19e9-0342-46e7-9071-61102051d033 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:17.102110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-58ab19e9-0342-46e7-9071-61102051d033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:17.180333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 448/875] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:17.202415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a383fbf-f482-4026-ad2c-b2032ec8b6bc None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:17.312830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a383fbf-f482-4026-ad2c-b2032ec8b6bc None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:17.346929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:17.347745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.348022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.348290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.348974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:17.350219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9136bb2-43cf-436b-94db-0df5208f957b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.350330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:17.350552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9136bb2-43cf-436b-94db-0df5208f957b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.350772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9136bb2-43cf-436b-94db-0df5208f957b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.356181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 449/876] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:17.358061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175a78-d59e-44bf-8d4b-c36bddab2470 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.358374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175a78-d59e-44bf-8d4b-c36bddab2470 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.358653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175a78-d59e-44bf-8d4b-c36bddab2470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:17.363024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:17.363999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:17.365239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 428/877] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:17.372478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:17.375969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-56175a78-d59e-44bf-8d4b-c36bddab2470 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:17.377231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-56175a78-d59e-44bf-8d4b-c36bddab2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:17.447516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 450/878] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:17.448980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396551b-850b-409f-9b60-d9004bb48f1e None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.449263 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396551b-850b-409f-9b60-d9004bb48f1e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.452815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396551b-850b-409f-9b60-d9004bb48f1e None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.452815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 451/879] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:17.456019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439ea08-aef4-4709-9cc7-ffbac9eba64d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.456302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439ea08-aef4-4709-9cc7-ffbac9eba64d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.456575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439ea08-aef4-4709-9cc7-ffbac9eba64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.472935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f439ea08-aef4-4709-9cc7-ffbac9eba64d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:17.473276 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f041d-9d66-4948-b8c1-e12a933fa015 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:17.474109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f439ea08-aef4-4709-9cc7-ffbac9eba64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:17.506632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:17.507302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.507585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.507931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.508644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:17.508978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:17.510351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:17.515055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 429/880] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:17.515804 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:17.541127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 452/881] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:17.541881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439ea08-aef4-4709-9cc7-ffbac9eba64d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:17.617726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392203c-5a35-439f-ab33-e2b727a05654 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:17.640222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439ea08-aef4-4709-9cc7-ffbac9eba64d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:17.650878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:17.651542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.651816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.652084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.652826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:17.654183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:17.670181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 430/882] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:17 2022] POST /identity/v3/services => generated 269 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:17.670989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:17.690332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:17.692143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.692968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.693468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.696323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:17.696957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:17.699224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:17.706147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 453/883] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:17.801409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e509e74-02ae-45b1-ba3c-54221f1d1b3e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:17.841225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:17.842485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:17.842967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:17.843451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:17.856654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:17.865460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:17.866541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:17.867479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:17.870647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 431/884] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 4 14:39:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:17.877233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:17.986870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1312-0ce5-426a-bd65-1b2960f6ec96 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:18.020958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:18.021693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:18.022375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:18.022649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:18.023476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:18.023786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:18.025182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:18.034374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 454/885] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 4 14:39:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:18.039944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:18.144143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b435e-7b7a-4b78-a359-a995fa24d1a6 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:18.179071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:18.180328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:18.180841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:18.181299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:18.192701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:18.194502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:18.212219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:18.213192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:18.214187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:18.214187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:18.214187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:18.214187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:18.216405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 432/886] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:18.222230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:18.327176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be226b1-ed19-4250-b488-d1b3e7fe6504 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:18.361303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:18.361980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:18.362260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:18.362531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:18.363259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:18.363562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:18.364965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:18.377596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 455/887] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:18.383359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:18.483127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80560159-71c5-4636-a463-7824eb94a0d2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:18.517238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:18.518453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:18.518945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:18.519432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:18.528319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:18.530111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:18.545704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:18.546711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:18.547644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:18.550314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 433/888] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:39:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:18.556228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:18.669167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4cdaa-fe02-4137-89f7-ae287de973c3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:18.709738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:18.710939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:18.711463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:18.711956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:18.713136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:18.713655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:18.715348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:18.737320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 456/889] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 4 14:39:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:18.743604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:18.851670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171fac6-967c-4163-ae2e-9f2016e61e45 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:18.888209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e51af6-b19e-4972-9db3-804dd667e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:18.889357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51af6-b19e-4972-9db3-804dd667e5a0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:18.889834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51af6-b19e-4972-9db3-804dd667e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:18.890298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e51af6-b19e-4972-9db3-804dd667e5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:18.964853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e51af6-b19e-4972-9db3-804dd667e5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:18.966646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e51af6-b19e-4972-9db3-804dd667e5a0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.073134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 434/890] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:18 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:19.075155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.097465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca7c78-d568-4540-b153-0b1289dd5a38 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.098095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca7c78-d568-4540-b153-0b1289dd5a38 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.099024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca7c78-d568-4540-b153-0b1289dd5a38 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.101281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 435/891] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:19.106664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06493bb7-f35c-412e-a0f4-775ca22437ec None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.107233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06493bb7-f35c-412e-a0f4-775ca22437ec None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.107857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06493bb7-f35c-412e-a0f4-775ca22437ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.129456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-06493bb7-f35c-412e-a0f4-775ca22437ec None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:19.131288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-06493bb7-f35c-412e-a0f4-775ca22437ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:19.187253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309455a0-c477-4adf-a205-e90e147d9766 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.225124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 436/892] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:19.229034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.230127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.230475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.230808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.241300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.242934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.243512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06493bb7-f35c-412e-a0f4-775ca22437ec None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:19.264836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:19.265834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:19.266731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:19.266731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:19.266731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:19.266731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:19.267423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 457/893] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:19.272709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.355254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06493bb7-f35c-412e-a0f4-775ca22437ec None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.370125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff867b-3a80-44da-9c70-70a2c9ed2369 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.389485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.390543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.390970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.391408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.392207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.393778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.404033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.404803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.405091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.405365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.406099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:19.406405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.407774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:19.409034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:19.410203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:19.411243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:19.419827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 437/894] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:19.421783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.424621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 458/895] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:19.430224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.532041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079850b8-5873-4dde-913d-cf66b055401f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.551474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32257-f247-4a20-9a79-e687c20ceed8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.566439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.567149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.567463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.567726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.568439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:19.568739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.570158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.575641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 438/896] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:19.581382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.596193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.597733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.598019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.598288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.608781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.611090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:19.630503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:19.631632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:19.632818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:19.635644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 459/897] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:19.641104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.696131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0216d-fa5e-493e-9d95-b242482f088a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.732710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.733600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.733941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.734274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.735056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:19.735427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.737757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.748166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 439/898] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:19.751980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.774070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f80ee7-6f39-45a5-b1f9-12adc8ab9cee None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.817868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.819252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.819720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.820198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.821447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:19.822074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.824331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.843441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 460/899] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:19.844598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.882570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0daad2f-fe45-4c9d-928e-bab7a0c1d5df None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:19.924905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:19.926170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:19.926911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:19.927335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:19.928462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] RBAC: Adding query filter params (interface=public, service_id=6e063c7f98434426ac07ae7acca3e931, region_id=RegionOne) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:19.928976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:19.931315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:19.946309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 440/900] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/endpoints?service_id=6e063c7f98434426ac07ae7acca3e931&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:19.947393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:19.952184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86dce7-2e48-42fb-b358-f4c22f1a71b5 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:20.000745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75780b61-604c-4a1e-9c08-f6b8e427ecbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:20.002061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75780b61-604c-4a1e-9c08-f6b8e427ecbf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.002504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75780b61-604c-4a1e-9c08-f6b8e427ecbf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.002918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75780b61-604c-4a1e-9c08-f6b8e427ecbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.003966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75780b61-604c-4a1e-9c08-f6b8e427ecbf None admin] RBAC: Adding query filter params (role.id=a5413484a9f84016bf6ffe3510bb7f75, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=b32490f3e59845f0a2720ca879664b21) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:20.004423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75780b61-604c-4a1e-9c08-f6b8e427ecbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:20.006848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75780b61-604c-4a1e-9c08-f6b8e427ecbf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:20.015539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 461/901] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/role_assignments?user.id=b32490f3e59845f0a2720ca879664b21&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=a5413484a9f84016bf6ffe3510bb7f75 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:20.034276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e6a9-ac67-4ace-a9b2-3d5dc8cac004 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.034612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e6a9-ac67-4ace-a9b2-3d5dc8cac004 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.034833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e6a9-ac67-4ace-a9b2-3d5dc8cac004 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.040674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 462/902] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:20.042780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecac086-64f3-4d63-bda9-2b881e899235 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.043066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecac086-64f3-4d63-bda9-2b881e899235 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.043345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecac086-64f3-4d63-bda9-2b881e899235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.061389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-7ecac086-64f3-4d63-bda9-2b881e899235 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:20.062603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7ecac086-64f3-4d63-bda9-2b881e899235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:20.072181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f7e2e-9de8-401d-8871-d3a29a582082 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:20.120368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23f8a4-4af2-48ab-a58d-8a07a0742cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:20.121822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23f8a4-4af2-48ab-a58d-8a07a0742cb1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.122312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23f8a4-4af2-48ab-a58d-8a07a0742cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.122815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23f8a4-4af2-48ab-a58d-8a07a0742cb1 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.123724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23f8a4-4af2-48ab-a58d-8a07a0742cb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:20.126286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23f8a4-4af2-48ab-a58d-8a07a0742cb1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:20.135271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 441/903] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:19 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:39:20.149596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 463/904] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:20.164742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f4d8-e638-41ce-a68e-52facca08d46 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.165461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f4d8-e638-41ce-a68e-52facca08d46 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.165940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80f4d8-e638-41ce-a68e-52facca08d46 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.167025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecac086-64f3-4d63-bda9-2b881e899235 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:20.168292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 442/905] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:20.174784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae6d2c-286c-49b2-9d8d-15681a9c13a5 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.175267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae6d2c-286c-49b2-9d8d-15681a9c13a5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.175716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae6d2c-286c-49b2-9d8d-15681a9c13a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.194951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c6ae6d2c-286c-49b2-9d8d-15681a9c13a5 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:20.196255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c6ae6d2c-286c-49b2-9d8d-15681a9c13a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:20.279608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 443/906] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:20.297076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecac086-64f3-4d63-bda9-2b881e899235 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:20.302599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae6d2c-286c-49b2-9d8d-15681a9c13a5 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:20.348296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:20.349207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.349519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.349761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.350479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:20.352025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:20.365256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:20.366266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:20.367416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 464/907] 10.222.0.32 () {56 vars in 1132 bytes} [Wed May 4 14:39:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:20.372425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:20.421585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae6d2c-286c-49b2-9d8d-15681a9c13a5 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:20.463365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:20.464474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.465011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.465489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.466418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:20.468570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:20.490074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 444/908] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:20 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:20.501983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce0aa7-7b75-4a17-abbc-9a7842e624c6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:20.549046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:20.549824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.550164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.550442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.551175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:20.551483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:20.552979 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:20.559621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 465/909] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:20.566006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:20.692873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad300be3-62fd-4a89-9905-12fdba4be952 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:20.726662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:20.727341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.727622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.727896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.728616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:20.728952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:20.730343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:20.735812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 445/910] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:20.741647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:20.844577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4da08c-200c-462d-b086-5f68f0ab2de3 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:20.879961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c22d7-3fa2-488d-90e1-d970435d10a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:20.881184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c22d7-3fa2-488d-90e1-d970435d10a7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:20.881656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c22d7-3fa2-488d-90e1-d970435d10a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:20.882107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c22d7-3fa2-488d-90e1-d970435d10a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:20.883206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c22d7-3fa2-488d-90e1-d970435d10a7 None admin] RBAC: Adding query filter params (interface=public, service_id=fbeb8e86170844e3b50fd3d9f143d0bb, region_id=RegionOne) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:20.883693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c22d7-3fa2-488d-90e1-d970435d10a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:20.885345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c22d7-3fa2-488d-90e1-d970435d10a7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:20.891495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 466/911] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:39:20 2022] GET /identity/v3/endpoints?service_id=fbeb8e86170844e3b50fd3d9f143d0bb&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:20.897188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:21.002733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffced3-17de-4f48-bf9d-ee02c33542ab None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:21.037131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61576f-0ce5-4ef8-81db-9ddec492113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:21.038156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61576f-0ce5-4ef8-81db-9ddec492113a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:21.038618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61576f-0ce5-4ef8-81db-9ddec492113a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:21.039102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61576f-0ce5-4ef8-81db-9ddec492113a None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:21.039963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61576f-0ce5-4ef8-81db-9ddec492113a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:21.041529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61576f-0ce5-4ef8-81db-9ddec492113a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:21.048431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 446/912] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:20 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:39:22.326368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95270891-7d0a-4d45-86e6-3fddce82a2e0 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.326923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95270891-7d0a-4d45-86e6-3fddce82a2e0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.326923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95270891-7d0a-4d45-86e6-3fddce82a2e0 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.328772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 467/913] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:22.335208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603061-d7e1-49f5-b024-43a313264783 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.335822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603061-d7e1-49f5-b024-43a313264783 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.335822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603061-d7e1-49f5-b024-43a313264783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.354409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-2e603061-d7e1-49f5-b024-43a313264783 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:22.355620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2e603061-d7e1-49f5-b024-43a313264783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:22.432150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 447/914] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:22.454325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95270891-7d0a-4d45-86e6-3fddce82a2e0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:22.520061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adda14-35a1-4562-93f5-8925c4a85661 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.520407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adda14-35a1-4562-93f5-8925c4a85661 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.520624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03adda14-35a1-4562-93f5-8925c4a85661 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.522735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 448/915] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:22.528312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d308-96cb-40c7-8509-2290b17054d1 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.528617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d308-96cb-40c7-8509-2290b17054d1 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.528892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6d308-96cb-40c7-8509-2290b17054d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.546746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-04e6d308-96cb-40c7-8509-2290b17054d1 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:22.547957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-04e6d308-96cb-40c7-8509-2290b17054d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:22.583314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95270891-7d0a-4d45-86e6-3fddce82a2e0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:22.624894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 449/916] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:22.627628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:22.628735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.629268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.629742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.641959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:22.643329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d308-96cb-40c7-8509-2290b17054d1 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:22.653361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:22.655266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:22.655925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 468/917] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 4 14:39:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:22.665908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:22.758395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6d308-96cb-40c7-8509-2290b17054d1 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:22.780463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904aa435-3b7d-44ac-8a9d-09f1f76d906b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:22.805268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:22.806529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.807107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.807582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.808495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:22.810132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:22.816586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:22.817347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.817627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.817899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.818709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:22.819064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:22.820524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:22.824810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:22.825823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 04 14:39:22.826676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:22.832843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 469/919] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 4 14:39:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:22.834229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 450/919] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:22.835452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:22.835647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:22.939486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1f080-7647-4ed8-9638-eaadba61d299 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:22.946408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9f97-a795-4ac9-a552-877fafcc9afc None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:22.975413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce7034-8ab6-4cae-b28b-e3b87d98147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:22.976435 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce7034-8ab6-4cae-b28b-e3b87d98147c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.976748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce7034-8ab6-4cae-b28b-e3b87d98147c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.977060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce7034-8ab6-4cae-b28b-e3b87d98147c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.985115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542292d-881c-4c24-ac02-ef587112f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:22.986114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542292d-881c-4c24-ac02-ef587112f9dc None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.986392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542292d-881c-4c24-ac02-ef587112f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:22.986671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542292d-881c-4c24-ac02-ef587112f9dc None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:22.987409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542292d-881c-4c24-ac02-ef587112f9dc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:22.987740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542292d-881c-4c24-ac02-ef587112f9dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:22.988532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce7034-8ab6-4cae-b28b-e3b87d98147c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:22.989125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542292d-881c-4c24-ac02-ef587112f9dc None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:22.990030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce7034-8ab6-4cae-b28b-e3b87d98147c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:22.997152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 451/920] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:22.999341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6e1e-c161-49e8-9e24-6285be73be62 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:22.999905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6e1e-c161-49e8-9e24-6285be73be62 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.000375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6e1e-c161-49e8-9e24-6285be73be62 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.001944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 452/921] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:23.002880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6e1e-c161-49e8-9e24-6285be73be62 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-12ce7034-8ab6-4cae-b28b-e3b87d98147c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:23.008341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:23.009373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:23.010277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:23.010277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:23.010277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:23.010277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:23.010277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 470/922] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:23.011273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04873-9867-41d9-a3a7-11129045be38 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.011561 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04873-9867-41d9-a3a7-11129045be38 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.011835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a04873-9867-41d9-a3a7-11129045be38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.029949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-47a04873-9867-41d9-a3a7-11129045be38 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:23.031123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-47a04873-9867-41d9-a3a7-11129045be38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:23.113449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 471/923] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:23.113449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04873-9867-41d9-a3a7-11129045be38 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.132672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6e1e-c161-49e8-9e24-6285be73be62 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.170771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.171468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.171750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.172023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.173037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:23.173688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.176036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.182493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 453/924] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:23.183435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.229461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a04873-9867-41d9-a3a7-11129045be38 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.264078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.264766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.265072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.265341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.266073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:23.266377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.267780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.281343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 472/925] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:23.286756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.298895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9d94-78b9-40fc-a8cb-a500ef7d7e0f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.335892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.336722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.337063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.337346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.338010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.339396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:23.352595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:23.353697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:23.360063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 454/926] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:23.360909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.415318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b96df-8d6b-4a75-810c-1cb078aaf9f9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.454317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.454991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.455272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.455544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.456291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.457715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.485349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 473/927] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:23.485574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46254ade-9c37-4d30-bd1b-3703160e266a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.485660 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.562103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e1144-97a4-441f-82a5-becf6aa878d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.564090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e1144-97a4-441f-82a5-becf6aa878d5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.564391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e1144-97a4-441f-82a5-becf6aa878d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.564699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e1144-97a4-441f-82a5-becf6aa878d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.574995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e1144-97a4-441f-82a5-becf6aa878d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.576469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e1144-97a4-441f-82a5-becf6aa878d5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-851e1144-97a4-441f-82a5-becf6aa878d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:23.592709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:23.593764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:23.594744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:23.596985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 455/928] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:23.600268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f97c6-73cd-4b86-9f19-84eabfc85f79 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.600558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f97c6-73cd-4b86-9f19-84eabfc85f79 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.600858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f97c6-73cd-4b86-9f19-84eabfc85f79 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.602526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 456/929] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:23.603213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f97c6-73cd-4b86-9f19-84eabfc85f79 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.629823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033680-2607-4521-af61-da6b93243a52 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.678218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363a281-f168-43bf-b6ba-4a541714f3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.678649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363a281-f168-43bf-b6ba-4a541714f3b1 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.678926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363a281-f168-43bf-b6ba-4a541714f3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.679200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363a281-f168-43bf-b6ba-4a541714f3b1 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.680094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363a281-f168-43bf-b6ba-4a541714f3b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:23.680399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363a281-f168-43bf-b6ba-4a541714f3b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.681851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363a281-f168-43bf-b6ba-4a541714f3b1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.688192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 474/930] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:23.689574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9c3a2-6afd-416b-ba1d-0187f6834d19 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.689860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9c3a2-6afd-416b-ba1d-0187f6834d19 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.690122 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9c3a2-6afd-416b-ba1d-0187f6834d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.708401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b8a9c3a2-6afd-416b-ba1d-0187f6834d19 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:23.709542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b8a9c3a2-6afd-416b-ba1d-0187f6834d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:23.736036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f97c6-73cd-4b86-9f19-84eabfc85f79 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.779738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.780542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.780784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.781126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.781953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:23.782258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.783701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.791684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 475/931] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:23.793113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9c3a2-6afd-416b-ba1d-0187f6834d19 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.807678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 457/932] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:23.808848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.920019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9c3a2-6afd-416b-ba1d-0187f6834d19 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.923947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf02cef-33e2-46aa-a6c2-5eb2d4ab1dcc None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:23.963087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.963763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.964063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.964411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:23.964523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.965280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:23.965556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:23.965757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:23.965852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:23.966286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.968715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:23.979371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 476/933] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:23.979371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:23.985767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:23.987427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:24.006510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:24.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:24.008426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:24.008426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:24.008426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:24.008426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:24.008426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 458/934] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:24.008724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.092844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f29b32-2826-469a-ac8c-77b557d305ee None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.125080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a0c78-d10b-4299-afe0-c77a5ae7dc21 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.132999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.133799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.134076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.134392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.135005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.136740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:24.153176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:24.158938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:24.160482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 477/935] 10.222.0.32 () {56 vars in 1132 bytes} [Wed May 4 14:39:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:24.161578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.172653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.173497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.173842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.174179 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.175057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] RBAC: Adding query filter params (interface=public, service_id=0b3406aedd8a47b5b1e6b12e1e703a2a, region_id=RegionOne) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:24.175459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.177680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.184515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 459/936] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:39:24 2022] GET /identity/v3/endpoints?service_id=0b3406aedd8a47b5b1e6b12e1e703a2a&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:24.185932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.284840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbe424-d542-4bd3-b5aa-1224c26793e8 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.309686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7272-201f-4b80-b095-d73ca89634e1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.330298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.330990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.331247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.331513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.332432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:24.332737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.334166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.348700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.349261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 478/937] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:24.349769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.353582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.354172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.354439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.355189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:24.355494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.356866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.362148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 460/938] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:24.363148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.458306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05e548-7401-4b19-85bb-aa3a7c53a768 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.475720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc2a68-1d8e-4e22-8146-bd8e72b62577 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.493011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.493695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.493972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.494244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.495257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.496662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.502931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 479/939] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:24 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:39:24.503738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.522977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.524537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.525298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.525765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.535920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.538486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:24.556998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:24.557983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:24.558882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:24.561952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 461/940] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:24.568108 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.641658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2e35c-21f1-491d-808f-af3b68c55f80 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.692954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.694063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.694360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.694635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.695458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:24.695769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.697223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.708446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 480/941] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:24.709646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.724316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e0c47-5305-4610-a351-0f5f63efd11d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.765479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.766380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.766611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.766963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.767830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:24.768143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.770346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.790775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 462/942] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:24.792016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:24.836866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fe870-5b11-4b98-b775-c91bcc96668b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.879505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cc2a3-e5ce-452d-8882-b65489f7328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.880194 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cc2a3-e5ce-452d-8882-b65489f7328c None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.880469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cc2a3-e5ce-452d-8882-b65489f7328c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.880743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cc2a3-e5ce-452d-8882-b65489f7328c None admin] PATH_INFO: `/v3/endpoints` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.881547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cc2a3-e5ce-452d-8882-b65489f7328c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.883567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cc2a3-e5ce-452d-8882-b65489f7328c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.911033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 481/943] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:24.911872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412d98f-cd17-4c41-8013-81c2b2fb1550 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:24.968986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de761c-296a-4873-91a1-90beb584613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:24.970266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de761c-296a-4873-91a1-90beb584613c None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:24.970726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de761c-296a-4873-91a1-90beb584613c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:24.971175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de761c-296a-4873-91a1-90beb584613c None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/b32490f3e59845f0a2720ca879664b21/roles/a5413484a9f84016bf6ffe3510bb7f75` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:24.974257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de761c-296a-4873-91a1-90beb584613c None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=b32490f3e59845f0a2720ca879664b21, role_id=a5413484a9f84016bf6ffe3510bb7f75)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:24.976389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de761c-296a-4873-91a1-90beb584613c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:24.994059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 463/944] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:39:24 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/b32490f3e59845f0a2720ca879664b21/roles/a5413484a9f84016bf6ffe3510bb7f75 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:39:25.875648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a4bf0-4f6c-4776-8033-09ada3f7e0ee None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:25.876189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a4bf0-4f6c-4776-8033-09ada3f7e0ee None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:25.876189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a4bf0-4f6c-4776-8033-09ada3f7e0ee None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:25.877951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 482/945] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:25.883972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1615-d470-46af-85e5-e71fdaf2e69e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:25.884483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1615-d470-46af-85e5-e71fdaf2e69e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:25.884483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1615-d470-46af-85e5-e71fdaf2e69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:25.902019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e29b1615-d470-46af-85e5-e71fdaf2e69e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:25.903201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e29b1615-d470-46af-85e5-e71fdaf2e69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:25.970142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 464/946] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:25.993555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a4bf0-4f6c-4776-8033-09ada3f7e0ee None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:26.102983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a4bf0-4f6c-4776-8033-09ada3f7e0ee None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:26.140206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:26.140804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:26.141090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:26.141358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:26.142090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:26.143525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:26.144922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING keystone.server.flask.common [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 04 14:39:26.164527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.sql.core [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 04 14:39:26.164527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 04 14:39:26.164527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [parameters: {'id': '65a9c2e1062649578afc5de71498243e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 04 14:39:26.164527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91630) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65a9c2e1062649578afc5de71498243e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65a9c2e1062649578afc5de71498243e. May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result = self._query(query) May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application conn.query(q) May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 04 14:39:26.172834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result.read() May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application packet.raise_for_error() May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 04 14:39:26.173782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return project_ref.to_dict() May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application next(self.gen) May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application yield resource May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application next(self.gen) May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.session.rollback() May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.force_reraise() May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise self.value May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application session.commit() May 04 14:39:26.174725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._prepare_impl() May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.session.flush() May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._flush(objects) May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application compat.raise_( May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application flush_context.execute() May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 04 14:39:26.175601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rec.execute(self) May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application _emit_insert_statements( May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application c = connection._execute_20( May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ret = self._execute_context( May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 04 14:39:26.176540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self.dialect.do_execute( May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result = self._query(query) May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application conn.query(q) May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application result.read() May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 04 14:39:26.178814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application packet.raise_for_error() May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [parameters: {'id': '65a9c2e1062649578afc5de71498243e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:26.185099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:26.187189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.Conflict( May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65a9c2e1062649578afc5de71498243e. May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:26.189347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 483/947] 10.222.0.32 () {60 vars in 1140 bytes} [Wed May 4 14:39:25 2022] POST /identity/v3/projects => generated 262 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:26.190574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b1615-d470-46af-85e5-e71fdaf2e69e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:26.281186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b1615-d470-46af-85e5-e71fdaf2e69e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:26.315117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21879ed-ec9e-4d45-918c-6f1027f23622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:26.315988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21879ed-ec9e-4d45-918c-6f1027f23622 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:26.316265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21879ed-ec9e-4d45-918c-6f1027f23622 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:26.317002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21879ed-ec9e-4d45-918c-6f1027f23622 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:26.325206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21879ed-ec9e-4d45-918c-6f1027f23622 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:26.326737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21879ed-ec9e-4d45-918c-6f1027f23622 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-a21879ed-ec9e-4d45-918c-6f1027f23622 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:26.341097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:26.343194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:26.345288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 465/948] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:26.348600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:26.453354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b41061-1582-4d68-941d-1172631c82f7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:26.486745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7aaa-2c60-4463-ad96-84bd02082dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:26.487425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7aaa-2c60-4463-ad96-84bd02082dc1 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:26.487702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7aaa-2c60-4463-ad96-84bd02082dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:26.487973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7aaa-2c60-4463-ad96-84bd02082dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:26.488739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7aaa-2c60-4463-ad96-84bd02082dc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:26.489064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7aaa-2c60-4463-ad96-84bd02082dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:26.490735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7aaa-2c60-4463-ad96-84bd02082dc1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:26.512582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 484/949] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:26.901889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ad5f-9ab8-4ecc-89d4-3de2a8985027 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:26.902545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ad5f-9ab8-4ecc-89d4-3de2a8985027 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:26.902545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ad5f-9ab8-4ecc-89d4-3de2a8985027 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:26.904202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 466/950] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:26.909894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1d417-86ea-410f-be58-d8c120bf5572 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:26.910479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1d417-86ea-410f-be58-d8c120bf5572 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:26.910479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1d417-86ea-410f-be58-d8c120bf5572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:26.928740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c9b1d417-86ea-410f-be58-d8c120bf5572 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:26.930396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c9b1d417-86ea-410f-be58-d8c120bf5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:27.004968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 485/951] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:27.024051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ad5f-9ab8-4ecc-89d4-3de2a8985027 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.136395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ad5f-9ab8-4ecc-89d4-3de2a8985027 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.173616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae1cb3-6013-4eb6-8d3f-88cf52a5f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.174366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae1cb3-6013-4eb6-8d3f-88cf52a5f60b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.174650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae1cb3-6013-4eb6-8d3f-88cf52a5f60b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.174922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae1cb3-6013-4eb6-8d3f-88cf52a5f60b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.175583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae1cb3-6013-4eb6-8d3f-88cf52a5f60b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:27.176973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae1cb3-6013-4eb6-8d3f-88cf52a5f60b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-06ae1cb3-6013-4eb6-8d3f-88cf52a5f60b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:27.189455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:27.191901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:27.193904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:27.195838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 467/952] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:27.200837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1d417-86ea-410f-be58-d8c120bf5572 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.303508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1d417-86ea-410f-be58-d8c120bf5572 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.321853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf88f-c8b1-4871-982f-3151d4b2ad66 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.321853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf88f-c8b1-4871-982f-3151d4b2ad66 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.321853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf88f-c8b1-4871-982f-3151d4b2ad66 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.321853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 468/953] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:27.323871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d88c6e-0bff-4e0b-9b70-063be010f90c None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.324368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d88c6e-0bff-4e0b-9b70-063be010f90c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.324869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d88c6e-0bff-4e0b-9b70-063be010f90c None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.326343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 469/954] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:27.327653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12730b9b-e4c8-4c35-aca3-7dc435162596 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.328142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12730b9b-e4c8-4c35-aca3-7dc435162596 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.328647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12730b9b-e4c8-4c35-aca3-7dc435162596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.346238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0984f-9a01-45c1-82d9-734525eb98b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.347016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0984f-9a01-45c1-82d9-734525eb98b3 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.347356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0984f-9a01-45c1-82d9-734525eb98b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.347516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-12730b9b-e4c8-4c35-aca3-7dc435162596 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:27.347674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0984f-9a01-45c1-82d9-734525eb98b3 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.348415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0984f-9a01-45c1-82d9-734525eb98b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:27.348564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-12730b9b-e4c8-4c35-aca3-7dc435162596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:27.348873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0984f-9a01-45c1-82d9-734525eb98b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:27.350251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0984f-9a01-45c1-82d9-734525eb98b3 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:27.360627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 486/955] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:27.360627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5a5eb-511a-4648-bf2a-dee5d6ce2990 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.360627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5a5eb-511a-4648-bf2a-dee5d6ce2990 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.360627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5a5eb-511a-4648-bf2a-dee5d6ce2990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.373839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d4b5a5eb-511a-4648-bf2a-dee5d6ce2990 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:27.375031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d4b5a5eb-511a-4648-bf2a-dee5d6ce2990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:27.420042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 470/956] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:27.424920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12730b9b-e4c8-4c35-aca3-7dc435162596 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.450604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 487/957] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:27.451691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5a5eb-511a-4648-bf2a-dee5d6ce2990 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.527111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12730b9b-e4c8-4c35-aca3-7dc435162596 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.546253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5a5eb-511a-4648-bf2a-dee5d6ce2990 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.563801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.564739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.565046 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.565321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.566034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:27.566345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:27.567738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:27.572977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 471/958] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:27.578869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.580250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.581096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.581364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.581636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] PATH_INFO: `/v3/services/compute` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.582291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:27.583674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:27.595648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:27.596724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:27.597635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 488/959] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:27.598450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.683967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ce745-5eed-4614-9b14-e872cd32f67e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.699294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f6550-ea71-44b7-94e7-557939a1bd7f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.718652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.719531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.719809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.720074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.730807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:27.737656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.738316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.738592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.738858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:27.739444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:27.740357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:27.741578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:27.742319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:27.742390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 472/960] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:27.742390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.745143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:27.777069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 489/961] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:27.777069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.881055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ba426-2e95-486d-a47c-60577a7d6446 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.889014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a9e63-992a-4278-b04a-cd1f69c93fb5 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:27.917435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.918148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.918372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.918650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.919372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:27.919675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:27.921102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:27.927418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 473/962] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:27.929077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:27.929937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:27.930196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:27.930401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:27.931214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:27.931519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:27.933070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:27.934252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:27.944366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 490/963] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:27.950417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:28.040818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1634-1be9-441c-bdad-e902a05bca96 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:28.053684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cced2-3324-4261-9968-19d67bcf8b73 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:28.077203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:28.077896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.078180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.078460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.079208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:28.079527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:28.080990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:28.090832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:28.091704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.091981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.092261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.092951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 474/964] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:28.106025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:28.107606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:28.128629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:28.129645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:28.130541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:28.130541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:28.130541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:28.130541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:28.130541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 491/965] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:28.137356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:28.251459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95fe0e-e2cc-4c73-bf43-8d4e8e1f32ea None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:28.287822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:28.288499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.288858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.289138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.289879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:28.290189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:28.291563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:28.305300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 475/966] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:28.314310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:28.426538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fba53f-5fe8-4531-8056-3c3d6a4a4968 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:28.460877 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:28.462109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.462584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.463065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.471629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:28.473442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:28.489010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:28.490073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 04 14:39:28.491027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:28.493948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 492/967] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:39:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:28.499419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:28.602550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccd2c8-8e24-49ff-aa61-9c041d85dbb4 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:28.637434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99bf03-6db2-414a-87af-82ad0780605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:28.638144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99bf03-6db2-414a-87af-82ad0780605f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.638421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99bf03-6db2-414a-87af-82ad0780605f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.638690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99bf03-6db2-414a-87af-82ad0780605f None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.639550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99bf03-6db2-414a-87af-82ad0780605f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:28.639857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99bf03-6db2-414a-87af-82ad0780605f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:28.641348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99bf03-6db2-414a-87af-82ad0780605f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:28.662455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 476/968] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 4 14:39:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:28.669758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:28.777788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9407b-6b5f-43de-b9fb-8a97401162bd None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:28.811616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:28.812301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.812578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.812868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.858552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dae698-a387-46a1-b09e-488d186ec807 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.858728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dae698-a387-46a1-b09e-488d186ec807 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.859007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dae698-a387-46a1-b09e-488d186ec807 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.860513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 477/969] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:28.866292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de191-8da1-49da-b530-4625e129ab86 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:28.866589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de191-8da1-49da-b530-4625e129ab86 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:28.866890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887de191-8da1-49da-b530-4625e129ab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:28.882128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:28.883600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-887de191-8da1-49da-b530-4625e129ab86 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:28.883821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:28.884786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-887de191-8da1-49da-b530-4625e129ab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:28.966217 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 478/970] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:28.992163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de191-8da1-49da-b530-4625e129ab86 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:28.999706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 493/971] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:28 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:29.004944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:29.092396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887de191-8da1-49da-b530-4625e129ab86 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.101058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85c43-d3c9-4416-9243-9f7c406948f4 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.127015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.127642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.127919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.128192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.129018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.130511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.131848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING keystone.server.flask.common [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 04 14:39:29.136413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.137311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.137587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.137856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.145427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.sql.core [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 04 14:39:29.145427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 04 14:39:29.145427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [parameters: {'user_id': '7ad1cca9944a476e8a0e5f5099e93a67', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 04 14:39:29.145427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 04 14:39:29.148972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.150525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self.dialect.do_execute( May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application result = self._query(query) May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application conn.query(q) May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application result.read() May 04 14:39:29.162641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application packet.raise_for_error() May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 04 14:39:29.164779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application next(self.gen) May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application yield resource May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application next(self.gen) May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self.session.rollback() May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self.force_reraise() May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise self.value May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application session.commit() May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 04 14:39:29.167111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._prepare_impl() May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self.session.flush() May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._flush(objects) May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application compat.raise_( May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application flush_context.execute() May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rec.execute(self) May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 04 14:39:29.169246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application _emit_insert_statements( May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application result = connection._execute_20( May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ret = self._execute_context( May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 04 14:39:29.177274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self.dialect.do_execute( May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application result = self._query(query) May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application conn.query(q) May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application result.read() May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 04 14:39:29.179102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application packet.raise_for_error() May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [parameters: {'user_id': '7ad1cca9944a476e8a0e5f5099e93a67', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:29.189037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 04 14:39:29.190735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 04 14:39:29.192522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 04 14:39:29.192522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 04 14:39:29.192522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 04 14:39:29.192522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 04 14:39:29.192522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 04 14:39:29.192522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:29.192522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 479/972] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 4 14:39:28 2022] POST /identity/v3/users => generated 160 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:29.193342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:29.195054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:29.197008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:29.197008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:29.197008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:29.197008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:29.197440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:29.197551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 494/973] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:29.197551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:29.275384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4c5ae-839d-450e-93e2-e9d5b0392e64 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.279912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2a47e-b8bf-48d4-9926-922a419293ac None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.313726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.314422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.314729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.314998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.315801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:29.316107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.321024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.322161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.324913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.325195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.325467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.334641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.336085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.343146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 495/974] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:29.344347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:29.351380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:29.352352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 04 14:39:29.353345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:29.357444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 480/975] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:29.358322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:29.445879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da6fa2-547a-4e64-93bf-cfe71c855fcc None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.456988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1af4-65e4-40fe-9bac-e8b76fcd859c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.479871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.481070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.481545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.482017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.488999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.489946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.490614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.490892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.491160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.491710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.491989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:29.492289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.493828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:29.506975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:29.508034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:29.508962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:29.515908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 496/976] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:29.517047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:29.532575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 481/977] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 4 14:39:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:29.626626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24336e9-6701-42af-9d36-f68eac8467c7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.665310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859b558-7734-4ac5-979b-d56d54b241da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.666376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859b558-7734-4ac5-979b-d56d54b241da None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.666850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859b558-7734-4ac5-979b-d56d54b241da None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.667343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859b558-7734-4ac5-979b-d56d54b241da None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.668436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859b558-7734-4ac5-979b-d56d54b241da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:29.668974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859b558-7734-4ac5-979b-d56d54b241da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.670646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859b558-7734-4ac5-979b-d56d54b241da None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.688844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 497/978] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:29.691406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:29.796884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b276bef-173d-4287-a6f1-ab70dd17a7ea None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:29.832306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf0d02-b7c4-4110-a252-a5009c142a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:29.833012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf0d02-b7c4-4110-a252-a5009c142a6d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:29.833295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf0d02-b7c4-4110-a252-a5009c142a6d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:29.833564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf0d02-b7c4-4110-a252-a5009c142a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:29.834407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf0d02-b7c4-4110-a252-a5009c142a6d None admin] RBAC: Adding query filter params (role.id=a5413484a9f84016bf6ffe3510bb7f75, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=b32490f3e59845f0a2720ca879664b21) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:29.834718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf0d02-b7c4-4110-a252-a5009c142a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:29.836231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf0d02-b7c4-4110-a252-a5009c142a6d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:29.843132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 482/979] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:29 2022] GET /identity/v3/role_assignments?user.id=b32490f3e59845f0a2720ca879664b21&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=a5413484a9f84016bf6ffe3510bb7f75 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:30.135957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c5721-fe3e-4d68-884b-da24ca542101 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.136453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c5721-fe3e-4d68-884b-da24ca542101 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.136453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c5721-fe3e-4d68-884b-da24ca542101 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.137876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 498/980] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:30.143428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1025b-f9f5-4b8d-a9fb-f476ce7d32b6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.143852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1025b-f9f5-4b8d-a9fb-f476ce7d32b6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.143955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1025b-f9f5-4b8d-a9fb-f476ce7d32b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.161672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0db1025b-f9f5-4b8d-a9fb-f476ce7d32b6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:30.163162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0db1025b-f9f5-4b8d-a9fb-f476ce7d32b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:30.236823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 483/981] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:30.260852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c5721-fe3e-4d68-884b-da24ca542101 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:30.385424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c5721-fe3e-4d68-884b-da24ca542101 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:30.423104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a4bad-14dd-487f-a38a-cd92c3240f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:30.424332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a4bad-14dd-487f-a38a-cd92c3240f0d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.424843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a4bad-14dd-487f-a38a-cd92c3240f0d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.425352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a4bad-14dd-487f-a38a-cd92c3240f0d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.426228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a4bad-14dd-487f-a38a-cd92c3240f0d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:30.427852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a4bad-14dd-487f-a38a-cd92c3240f0d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-ce0a4bad-14dd-487f-a38a-cd92c3240f0d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:30.441179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:30.442426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:30.443436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:30.445798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 499/982] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:30.452366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1025b-f9f5-4b8d-a9fb-f476ce7d32b6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:30.463645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1276-57f5-4b0c-9aae-386b263e5064 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.463955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1276-57f5-4b0c-9aae-386b263e5064 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.464192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1276-57f5-4b0c-9aae-386b263e5064 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.465972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 500/983] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:30.470891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202df2-ca33-4b40-a7e3-c64e03bf655b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.471184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202df2-ca33-4b40-a7e3-c64e03bf655b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.472069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202df2-ca33-4b40-a7e3-c64e03bf655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.488575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-17202df2-ca33-4b40-a7e3-c64e03bf655b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:30.489824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-17202df2-ca33-4b40-a7e3-c64e03bf655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:30.557536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1025b-f9f5-4b8d-a9fb-f476ce7d32b6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:30.557737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 501/984] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:30.579963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17202df2-ca33-4b40-a7e3-c64e03bf655b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:30.590678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:30.591347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.591626 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.591900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.592618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:30.592960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:30.594328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:30.599230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 484/985] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:30.604641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:30.676914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17202df2-ca33-4b40-a7e3-c64e03bf655b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:30.704747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90a62-1743-4b2f-aad3-c0eeda7e4c73 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:30.714286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:30.714958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.715238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.715500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.716226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:30.717632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:30.733299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 502/986] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:30 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:30.760506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:30.761286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.761572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.761847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.762622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:30.762933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:30.764402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:30.770909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 485/987] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:30.777326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:30.888600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8cd10-51f1-43d9-8bfa-531680082b0d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:30.930151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46340a81-737a-4710-9592-c3a0dc67ebdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:30.930812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46340a81-737a-4710-9592-c3a0dc67ebdc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:30.931086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46340a81-737a-4710-9592-c3a0dc67ebdc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:30.931351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46340a81-737a-4710-9592-c3a0dc67ebdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:30.932168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46340a81-737a-4710-9592-c3a0dc67ebdc None admin] RBAC: Adding query filter params (interface=admin, service_id=0b3406aedd8a47b5b1e6b12e1e703a2a, region_id=RegionOne) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:30.932484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46340a81-737a-4710-9592-c3a0dc67ebdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:30.933961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46340a81-737a-4710-9592-c3a0dc67ebdc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:30.943232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 503/988] 10.222.0.32 () {56 vars in 1249 bytes} [Wed May 4 14:39:30 2022] GET /identity/v3/endpoints?service_id=0b3406aedd8a47b5b1e6b12e1e703a2a&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:30.946299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:31.069148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248d629-519d-47ee-a0fd-ba52be805792 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:31.107354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bdaf3-6eec-4fe4-b576-20f983dad409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:31.108034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdaf3-6eec-4fe4-b576-20f983dad409 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:31.108321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdaf3-6eec-4fe4-b576-20f983dad409 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:31.108596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdaf3-6eec-4fe4-b576-20f983dad409 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:31.109321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bdaf3-6eec-4fe4-b576-20f983dad409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:31.110728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bdaf3-6eec-4fe4-b576-20f983dad409 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:31.124830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 486/989] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:30 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:39:32.159570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bf858-3a8c-4557-9405-dc704fc39f14 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:32.160124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bf858-3a8c-4557-9405-dc704fc39f14 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:32.160124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bf858-3a8c-4557-9405-dc704fc39f14 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:32.161479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 504/990] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:32.167052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ac14f-7803-44a4-a213-ebb2c2502961 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:32.167478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ac14f-7803-44a4-a213-ebb2c2502961 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:32.167553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ac14f-7803-44a4-a213-ebb2c2502961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:32.184848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ca7ac14f-7803-44a4-a213-ebb2c2502961 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:32.186051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ca7ac14f-7803-44a4-a213-ebb2c2502961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:32.253698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 487/991] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:32.276237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bf858-3a8c-4557-9405-dc704fc39f14 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:32.382392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bf858-3a8c-4557-9405-dc704fc39f14 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:32.418364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:32.419088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:32.419364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:32.419635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] PATH_INFO: `/v3/services/image` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:32.420288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:32.421705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:32.433455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:32.434471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:32.435360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:32.437189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 505/992] 10.222.0.32 () {56 vars in 1105 bytes} [Wed May 4 14:39:32 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:32.440714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ac14f-7803-44a4-a213-ebb2c2502961 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:32.539588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ac14f-7803-44a4-a213-ebb2c2502961 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:32.573348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a00c2d-4b7d-4b54-be1e-b73319684126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:32.573925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a00c2d-4b7d-4b54-be1e-b73319684126 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:32.574197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a00c2d-4b7d-4b54-be1e-b73319684126 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:32.574463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a00c2d-4b7d-4b54-be1e-b73319684126 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:32.575226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a00c2d-4b7d-4b54-be1e-b73319684126 None admin] RBAC: Adding query filter params (name=image) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:32.575522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a00c2d-4b7d-4b54-be1e-b73319684126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:32.576966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a00c2d-4b7d-4b54-be1e-b73319684126 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:32.581958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 488/993] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:32.587420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:32.684492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28832297-193c-4929-bdaf-8719357e5784 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:32.721272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ba39c-ac87-454a-9f95-e3eacf169c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:32.721947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba39c-ac87-454a-9f95-e3eacf169c04 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:32.722223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba39c-ac87-454a-9f95-e3eacf169c04 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:32.722488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ba39c-ac87-454a-9f95-e3eacf169c04 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:32.723192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ba39c-ac87-454a-9f95-e3eacf169c04 None admin] RBAC: Adding query filter params (type=image) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:32.723498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ba39c-ac87-454a-9f95-e3eacf169c04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:32.724900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ba39c-ac87-454a-9f95-e3eacf169c04 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:32.729649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 506/994] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:33.054857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3614ff3-c903-4d6d-8b2a-5cdd5796172b None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.055284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3614ff3-c903-4d6d-8b2a-5cdd5796172b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.055398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3614ff3-c903-4d6d-8b2a-5cdd5796172b None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.056845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 489/995] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:33.062268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd80f-a31b-4dfe-9bce-11dabdc88e51 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.062633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd80f-a31b-4dfe-9bce-11dabdc88e51 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.062983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd80f-a31b-4dfe-9bce-11dabdc88e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.080454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c7cbd80f-a31b-4dfe-9bce-11dabdc88e51 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:33.081684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c7cbd80f-a31b-4dfe-9bce-11dabdc88e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:33.151411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 507/996] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:33.174367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3614ff3-c903-4d6d-8b2a-5cdd5796172b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.280911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3614ff3-c903-4d6d-8b2a-5cdd5796172b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.325309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d2e32-4ee9-45ed-8243-03414f281eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.326312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d2e32-4ee9-45ed-8243-03414f281eb5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.326838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d2e32-4ee9-45ed-8243-03414f281eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.327249 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d2e32-4ee9-45ed-8243-03414f281eb5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.328064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d2e32-4ee9-45ed-8243-03414f281eb5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.329645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d2e32-4ee9-45ed-8243-03414f281eb5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-cf4d2e32-4ee9-45ed-8243-03414f281eb5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:33.342404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:33.343576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:33.344423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 490/997] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:33.352021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbd80f-a31b-4dfe-9bce-11dabdc88e51 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.404043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598b041-c66b-4992-a63f-60660d1b2645 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.404413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598b041-c66b-4992-a63f-60660d1b2645 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.404668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598b041-c66b-4992-a63f-60660d1b2645 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.406120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 491/998] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:33.411267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a6c8-59a3-4742-87a7-3077e131ede1 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.411514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a6c8-59a3-4742-87a7-3077e131ede1 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.411792 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a6c8-59a3-4742-87a7-3077e131ede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.429004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c243a6c8-59a3-4742-87a7-3077e131ede1 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:33.430321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c243a6c8-59a3-4742-87a7-3077e131ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:33.456667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbd80f-a31b-4dfe-9bce-11dabdc88e51 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.495492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.496175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.496455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.496729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.497476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:33.497784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.498585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 492/999] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:33.499169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.504222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 508/1000] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:33.510049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243a6c8-59a3-4742-87a7-3077e131ede1 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.522765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.607663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243a6c8-59a3-4742-87a7-3077e131ede1 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.622495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151980df-6404-4f59-8a90-4ea48dab62b2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.643114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.643795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.644075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.644349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.645122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:33.645429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.647235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.653099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 493/1001] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:33.659114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.661332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.662817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.663269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.663819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.664542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.665962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:33.678420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:33.679577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:33.680426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 509/1002] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:33.685453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.764222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be5467-7a0b-4bb5-bb3a-57d6f6cd85f5 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.783601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a662a1-3bb7-44c8-bc5a-e15f66eeb43f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.799674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.800350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.800631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.800936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.801744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] RBAC: Adding query filter params (interface=public, service_id=3890ae64fe2f4e6cb1be6497f6011d61, region_id=RegionOne) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:33.802067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.803426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.808675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 494/1003] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/endpoints?service_id=3890ae64fe2f4e6cb1be6497f6011d61&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:33.814311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.822231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.823335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.823619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.823893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.824997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:33.825308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.827319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.838551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 510/1004] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:33.839734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:33.915492 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93338d7b-2f96-4b5e-a555-b63fbc3fa8f2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.947421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a902e-63c5-43f6-a4e0-b15ed575a750 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:33.949058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.949761 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.950044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.950313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.951006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.952374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.964883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 495/1005] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:39:33.981268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1defa37-9f56-4262-99c1-529158890f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:33.981972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1defa37-9f56-4262-99c1-529158890f10 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:33.982257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1defa37-9f56-4262-99c1-529158890f10 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:33.982532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1defa37-9f56-4262-99c1-529158890f10 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:33.983267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1defa37-9f56-4262-99c1-529158890f10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:33.983574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1defa37-9f56-4262-99c1-529158890f10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:33.985033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1defa37-9f56-4262-99c1-529158890f10 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:33.995707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 511/1006] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 4 14:39:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:33.997156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:34.113951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e48c-ad46-480d-9ebc-501446996901 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:34.151739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d595f44-40c8-4f39-bd5b-436677e46488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:34.152367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d595f44-40c8-4f39-bd5b-436677e46488 None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:34.152667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d595f44-40c8-4f39-bd5b-436677e46488 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:34.152967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d595f44-40c8-4f39-bd5b-436677e46488 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:34.153704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d595f44-40c8-4f39-bd5b-436677e46488 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:34.155066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d595f44-40c8-4f39-bd5b-436677e46488 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:34.171738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 496/1007] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:34.970101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095a78-b631-42ca-af14-257dc7c5c99f None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:34.970652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095a78-b631-42ca-af14-257dc7c5c99f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:34.970652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095a78-b631-42ca-af14-257dc7c5c99f None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:34.971968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 512/1008] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:34.978656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9974a-fba8-46a7-81be-b1ff5cd2c6a6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:34.979124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9974a-fba8-46a7-81be-b1ff5cd2c6a6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:34.979708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9974a-fba8-46a7-81be-b1ff5cd2c6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:35.006764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-70a9974a-fba8-46a7-81be-b1ff5cd2c6a6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:35.009463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-70a9974a-fba8-46a7-81be-b1ff5cd2c6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:35.127499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 497/1009] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:35.150976 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095a78-b631-42ca-af14-257dc7c5c99f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:35.256389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095a78-b631-42ca-af14-257dc7c5c99f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:35.305144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684aa45-e086-49f4-b207-93360b68409f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:35.305888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684aa45-e086-49f4-b207-93360b68409f None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:35.306173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684aa45-e086-49f4-b207-93360b68409f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:35.308965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684aa45-e086-49f4-b207-93360b68409f None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:35.308965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684aa45-e086-49f4-b207-93360b68409f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:35.309331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684aa45-e086-49f4-b207-93360b68409f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:35.334279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 513/1010] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:35 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:39:36.246026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06688d3-97eb-448f-abdf-0f67c5993778 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.246707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06688d3-97eb-448f-abdf-0f67c5993778 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.246707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06688d3-97eb-448f-abdf-0f67c5993778 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.248106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 498/1011] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:36.253305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47ff4a-93ee-4a6f-89d3-3224418d239a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.253768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47ff4a-93ee-4a6f-89d3-3224418d239a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.253857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47ff4a-93ee-4a6f-89d3-3224418d239a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.270883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ee47ff4a-93ee-4a6f-89d3-3224418d239a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:36.272413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ee47ff4a-93ee-4a6f-89d3-3224418d239a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:36.358099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 514/1012] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:36 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:36.380982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06688d3-97eb-448f-abdf-0f67c5993778 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:36.482805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06688d3-97eb-448f-abdf-0f67c5993778 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:36.518345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:36.519184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.519426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.519731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] PATH_INFO: `/v3/services/compute` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.520417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:36.522283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:36.527199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb984b-c397-47b7-ab31-b5cd2b0652df None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.527520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb984b-c397-47b7-ab31-b5cd2b0652df None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.527758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb984b-c397-47b7-ab31-b5cd2b0652df None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.529393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 515/1013] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:36.534488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c8f2-3b5e-4f70-91bd-1cf0bfdd0b7d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.534770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c8f2-3b5e-4f70-91bd-1cf0bfdd0b7d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.535043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c8f2-3b5e-4f70-91bd-1cf0bfdd0b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:36.535262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:36.536304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:36.537270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 499/1014] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:36.542414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:36.556545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ea07c8f2-3b5e-4f70-91bd-1cf0bfdd0b7d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:36.557782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ea07c8f2-3b5e-4f70-91bd-1cf0bfdd0b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:36.636620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 516/1015] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:36 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:36.641794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2fa43-37e7-4dc2-b587-242bad9770ef None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:36.660863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c8f2-3b5e-4f70-91bd-1cf0bfdd0b7d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:36.676316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:36.677287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.677765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.678230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.679190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] RBAC: Adding query filter params (name=compute) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:36.679682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:36.681252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:36.691076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 500/1016] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:36.692093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:36.754340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c8f2-3b5e-4f70-91bd-1cf0bfdd0b7d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:36.788168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:36.789067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.789540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.790021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.791875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:36.793610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466da2d-0099-4ca4-92d0-77149887aafb None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:36.793807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:36.796205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 517/1017] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:36.801887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:36.828273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:36.828989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.829275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.829574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.830266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:36.830577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:36.831978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:36.837135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 501/1018] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 4 14:39:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:36.842524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:36.903640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087919f-241a-468b-a6f6-fa28564bb8f6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:36.937505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecde3ae-d84c-48ec-bba1-07b865488252 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:36.939911 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4696b-a485-43a8-af71-a139acb0086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:36.940588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4696b-a485-43a8-af71-a139acb0086a None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.940904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4696b-a485-43a8-af71-a139acb0086a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.941178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4696b-a485-43a8-af71-a139acb0086a None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.941918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4696b-a485-43a8-af71-a139acb0086a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:36.943382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4696b-a485-43a8-af71-a139acb0086a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:36.976844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 518/1019] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:39:36 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:36.977087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d294a62-ec2a-467f-8521-dacfe32c01ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:36.977087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294a62-ec2a-467f-8521-dacfe32c01ef None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:36.977087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294a62-ec2a-467f-8521-dacfe32c01ef None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:36.977087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294a62-ec2a-467f-8521-dacfe32c01ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:36.977454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d294a62-ec2a-467f-8521-dacfe32c01ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:36.977454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d294a62-ec2a-467f-8521-dacfe32c01ef None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:37.001012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 502/1020] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:37.648913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4801e-a41e-4de0-915e-1f6dfaae40e6 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:37.649427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4801e-a41e-4de0-915e-1f6dfaae40e6 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:37.649427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4801e-a41e-4de0-915e-1f6dfaae40e6 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:37.650706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 519/1021] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:37.656302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187d8d-1963-464c-8050-f992fa42c223 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:37.656778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187d8d-1963-464c-8050-f992fa42c223 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:37.657007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9187d8d-1963-464c-8050-f992fa42c223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:37.673825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a9187d8d-1963-464c-8050-f992fa42c223 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:37.674987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a9187d8d-1963-464c-8050-f992fa42c223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:37.746541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 503/1022] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:37.769213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4801e-a41e-4de0-915e-1f6dfaae40e6 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:37.868278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4801e-a41e-4de0-915e-1f6dfaae40e6 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:37.900376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:37.901108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:37.901384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:37.901656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] PATH_INFO: `/v3/services/image` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:37.902298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:37.903661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:37.914996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:37.916024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:37.916904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 520/1023] 10.222.0.32 () {56 vars in 1105 bytes} [Wed May 4 14:39:37 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:37.924055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9187d8d-1963-464c-8050-f992fa42c223 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:38.030303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9187d8d-1963-464c-8050-f992fa42c223 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:38.070996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:38.071700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:38.071950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:38.072248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:38.073017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] RBAC: Adding query filter params (name=image) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:38.073284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:38.074667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:38.080339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 504/1024] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:38.086390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:38.194023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629e0d8-d3fe-44d4-8b68-937137e186be None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:38.228693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:38.229509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:38.229737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:38.229966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:38.230707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] RBAC: Adding query filter params (type=image) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:38.231005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:38.232864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:38.238981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 521/1025] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:38.245136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:38.374586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f8118-377a-4845-bee2-4f426c6c4b90 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:38.413017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92282faa-07fa-4534-be88-d2670b9e05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:38.413771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92282faa-07fa-4534-be88-d2670b9e05cc None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:38.414055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92282faa-07fa-4534-be88-d2670b9e05cc None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:38.414339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92282faa-07fa-4534-be88-d2670b9e05cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:38.415158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92282faa-07fa-4534-be88-d2670b9e05cc None admin] RBAC: Adding query filter params (interface=public, service_id=148047e7ebf342c1af73875bd28162ae, region_id=RegionOne) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:38.415468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92282faa-07fa-4534-be88-d2670b9e05cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:38.416880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92282faa-07fa-4534-be88-d2670b9e05cc None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:38.422763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 505/1026] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:39:38 2022] GET /identity/v3/endpoints?service_id=148047e7ebf342c1af73875bd28162ae&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:38.428622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:38.554023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a7476-34b6-41d9-bfc1-88ec07774f98 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:38.600163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1c10d-977f-4443-9032-f55d65cb8f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:38.601013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1c10d-977f-4443-9032-f55d65cb8f76 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:38.601291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1c10d-977f-4443-9032-f55d65cb8f76 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:38.601561 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1c10d-977f-4443-9032-f55d65cb8f76 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:38.602269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1c10d-977f-4443-9032-f55d65cb8f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:38.603691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1c10d-977f-4443-9032-f55d65cb8f76 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:38.611891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 522/1027] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:39:38 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:39:39.301052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3b8d8-fa13-4024-b884-747fbda5ff2c None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.301846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3b8d8-fa13-4024-b884-747fbda5ff2c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.301846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3b8d8-fa13-4024-b884-747fbda5ff2c None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.303299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 506/1028] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:39.311432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2754af-4ffa-40b6-8b14-304b3b9d3b34 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.312827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2754af-4ffa-40b6-8b14-304b3b9d3b34 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.313279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2754af-4ffa-40b6-8b14-304b3b9d3b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.334778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-cb2754af-4ffa-40b6-8b14-304b3b9d3b34 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:39.336014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cb2754af-4ffa-40b6-8b14-304b3b9d3b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:39.395929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f044a-6655-4ecb-9fb5-93e7cdab9568 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.396950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f044a-6655-4ecb-9fb5-93e7cdab9568 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.397550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f044a-6655-4ecb-9fb5-93e7cdab9568 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.400235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 507/1029] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:39.405976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbade0-7ade-42a0-98bc-56567b4f30e2 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.406254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbade0-7ade-42a0-98bc-56567b4f30e2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.406530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbade0-7ade-42a0-98bc-56567b4f30e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.408250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 523/1030] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:39.432410 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-80bbade0-7ade-42a0-98bc-56567b4f30e2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:39.433734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-80bbade0-7ade-42a0-98bc-56567b4f30e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:39.449504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2754af-4ffa-40b6-8b14-304b3b9d3b34 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:39.518461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 508/1031] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:39.558970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbade0-7ade-42a0-98bc-56567b4f30e2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:39.589205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2754af-4ffa-40b6-8b14-304b3b9d3b34 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:39.638311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:39.639504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.639787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.640044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.641319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:39.642780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:39.644470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 524/1032] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:39.650824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:39.690184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbade0-7ade-42a0-98bc-56567b4f30e2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:39.731003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:39.731939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.732240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.732521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.746213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:39.754403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:39.756231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:39.758683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 509/1033] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 4 14:39:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:39.761074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208cea3-cc25-4951-ab1a-896e67e72b47 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:39.763924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:39.799429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:39.800388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.800876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.801322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.802381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:39.804197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:39.833093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 525/1034] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:39:39 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:39.906921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85696405-a983-4c8a-a789-a49868ece269 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:39.949279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:39.950150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:39.950566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:39.950963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:39.951899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:39.952348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:39.954061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:39.969662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 510/1035] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 4 14:39:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:39.971958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:40.089735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce234d9c-8e38-42f1-b08b-364c9cb5acfd None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:40.132538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:40.133403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:40.133686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:40.133956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:40.146202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:40.148120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:40.166872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:40.168648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:40.170335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:40.170335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:40.170335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:40.170335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:40.170335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 526/1036] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:40.174287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:40.302449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac835fd0-e1f4-48e4-b6fd-e6f0bcc908dc None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:40.365769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:40.366983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:40.367434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:40.367949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:40.369451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:40.370015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:40.372733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:40.400843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 511/1037] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:40.403875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:40.518087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a1b6f-dfa9-4771-877b-49fa705925b6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:40.564476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:40.565302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:40.565741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:40.565925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:40.581288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:40.582763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:40.598062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:40.601893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:40.606259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 527/1038] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:39:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:40.608427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:40.711943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e74eb9-e38f-4198-b073-51064db156aa None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:40.748459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:40.749241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:40.749531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:40.749844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:40.750717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:40.751035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:40.752532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:40.773653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 512/1039] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:39:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:40.780126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:40.963287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f8000-9f42-4594-9c69-0c247c864c19 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:41.025101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:41.026216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:41.026709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:41.027202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:41.159127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:41.161958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:41.348834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 528/1040] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:41.351688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:41.523810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285c587-e8bb-4ce9-8eef-c7a6b9eab3a0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:41.584945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:41.586239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:41.587343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:41.587792 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:41.612429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:41.615194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:41.650115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:41.651426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:41.652723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:41.652723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:41.652723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:41.652723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:41.654382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 513/1041] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:41.661226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:41.838115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d89b-4acd-4dad-a989-00603ade02d2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:41.894573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210c823-bec8-48fc-96d9-cd57024c99d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:41.895620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210c823-bec8-48fc-96d9-cd57024c99d5 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:41.896097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210c823-bec8-48fc-96d9-cd57024c99d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:41.896865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210c823-bec8-48fc-96d9-cd57024c99d5 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:41.897944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210c823-bec8-48fc-96d9-cd57024c99d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:41.898231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210c823-bec8-48fc-96d9-cd57024c99d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:41.900035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210c823-bec8-48fc-96d9-cd57024c99d5 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:41.919527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 529/1042] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:41.925981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:41.969896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24882f5-7282-4b04-8f62-4eb6feb05849 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:41.970197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24882f5-7282-4b04-8f62-4eb6feb05849 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:41.970540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24882f5-7282-4b04-8f62-4eb6feb05849 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:41.972083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 530/1043] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:41.977686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925dd5-ed4b-4f0c-837e-bba48300f9cb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:41.977958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925dd5-ed4b-4f0c-837e-bba48300f9cb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:41.978196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925dd5-ed4b-4f0c-837e-bba48300f9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:41.996862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-45925dd5-ed4b-4f0c-837e-bba48300f9cb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:41.998097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-45925dd5-ed4b-4f0c-837e-bba48300f9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:42.072694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 531/1044] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:42.096598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45925dd5-ed4b-4f0c-837e-bba48300f9cb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:42.105797 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944e461-2777-4164-a0c4-454d07ce2ed8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:42.167968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:42.169107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.169587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.170087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.183037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:42.185810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:42.211793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45925dd5-ed4b-4f0c-837e-bba48300f9cb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:42.212259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:42.213583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:42.215103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:42.221190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 514/1045] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:42.223258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:42.274546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:42.275375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.275671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.275935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] PATH_INFO: `/v3/services/image` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.276587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:42.278055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:42.291201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:42.293428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:42.295359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 532/1046] 10.222.0.32 () {56 vars in 1105 bytes} [Wed May 4 14:39:42 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:42.299100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:42.378673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a55c5-c101-4d38-bcef-c7b299556c76 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:42.441729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327ef21-a18f-4587-865c-43ee31bc28c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:42.442817 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ef21-a18f-4587-865c-43ee31bc28c6 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.443254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ef21-a18f-4587-865c-43ee31bc28c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.443698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327ef21-a18f-4587-865c-43ee31bc28c6 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.445123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327ef21-a18f-4587-865c-43ee31bc28c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:42.445624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327ef21-a18f-4587-865c-43ee31bc28c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:42.448169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5327ef21-a18f-4587-865c-43ee31bc28c6 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:42.468735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 515/1047] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:42.470063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e9f66-3186-4f7d-ba24-9ad29eacef71 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:42.475436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6145f5-fbfa-44e0-b239-f963cf2aa5a7 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.475436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6145f5-fbfa-44e0-b239-f963cf2aa5a7 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.475780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6145f5-fbfa-44e0-b239-f963cf2aa5a7 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.477211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 516/1048] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:42.477964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6145f5-fbfa-44e0-b239-f963cf2aa5a7 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:42.511685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cd079-0289-4da4-b745-b567c0dd6800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:42.512571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd079-0289-4da4-b745-b567c0dd6800 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.513002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd079-0289-4da4-b745-b567c0dd6800 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.513445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd079-0289-4da4-b745-b567c0dd6800 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.514919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cd079-0289-4da4-b745-b567c0dd6800 None admin] RBAC: Adding query filter params (name=image) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:42.515489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cd079-0289-4da4-b745-b567c0dd6800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:42.518092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cd079-0289-4da4-b745-b567c0dd6800 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:42.533063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 533/1049] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:42.533063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26d6c-75a2-4299-bc89-39c89f92ae74 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.533063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26d6c-75a2-4299-bc89-39c89f92ae74 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.533063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa26d6c-75a2-4299-bc89-39c89f92ae74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.557591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-afa26d6c-75a2-4299-bc89-39c89f92ae74 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:42.559600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-afa26d6c-75a2-4299-bc89-39c89f92ae74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:42.620030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6145f5-fbfa-44e0-b239-f963cf2aa5a7 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:42.651877 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 534/1050] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:42.652822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa26d6c-75a2-4299-bc89-39c89f92ae74 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:42.678835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:42.679837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.680288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.680744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.682166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] RBAC: Adding query filter params (role.id=a5413484a9f84016bf6ffe3510bb7f75, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=753f2914e5bc4e298b99529e230d9ee3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:42.682769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:42.684832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:42.704832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 517/1051] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:42 2022] GET /identity/v3/role_assignments?user.id=753f2914e5bc4e298b99529e230d9ee3&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=a5413484a9f84016bf6ffe3510bb7f75 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:42.704832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:42.817487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa26d6c-75a2-4299-bc89-39c89f92ae74 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:42.826807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724f9b-fcdb-426c-976e-b6c43835f854 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:42.875651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:42.876543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.876858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.877141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.877899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] RBAC: Adding query filter params (type=image) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:42.878212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:42.878796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:42.879677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:42.880204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:42.880660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:42.881152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:42.886056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 535/1052] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:42.893108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:42.902483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:42.915854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:42.917544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:42.922106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 518/1053] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:42.927607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:43.038502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f58359-fb1c-4830-9c73-ad4cab0dde85 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:43.098949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:43.099961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:43.100381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:43.100918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:43.102107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98808e9-4095-4763-931e-850d7feced8d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:43.102269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:43.104379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:43.144829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 536/1054] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:39:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:43.178529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:43.179691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:43.180139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:43.180637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:43.182229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:43.182786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:43.185485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:43.200663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 519/1055] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:43.208570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:43.384243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed74b7-fd00-4fb7-b192-35a437a81b78 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:43.442230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:43.443056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:43.443340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:43.443599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] PATH_INFO: `/v3/users/88008d0c94b14b0ba660fe163dc11662` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:43.459568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] RBAC: Authorizing `identity:get_user(user_id=88008d0c94b14b0ba660fe163dc11662)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:43.461147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:43.463230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 537/1056] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:39:43 2022] GET /identity/v3/users/88008d0c94b14b0ba660fe163dc11662 => generated 285 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:43.469939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:43.645165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cabbc1-a818-4413-b852-4b6e6f2ea332 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:43.705381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:43.706366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:43.706820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:43.707382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:43.812689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:43.814280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:43.979680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 520/1057] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:43.982436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:44.116528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac7010-1ec5-4386-a468-d205cdbabb17 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:44.160757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53700fb6-9f31-41f1-97da-133c407a0217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:44.161741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53700fb6-9f31-41f1-97da-133c407a0217 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:44.162085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53700fb6-9f31-41f1-97da-133c407a0217 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:44.162405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53700fb6-9f31-41f1-97da-133c407a0217 None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:44.163762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53700fb6-9f31-41f1-97da-133c407a0217 None admin] RBAC: Authorizing `identity:get_project(project_id=5764822dbac4490fa04caf5f1898050d)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:44.165265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53700fb6-9f31-41f1-97da-133c407a0217 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:44.167346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 538/1058] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:39:43 2022] GET /identity/v3/projects/5764822dbac4490fa04caf5f1898050d => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:44.174469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:44.332449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de75a1-09da-418c-af6b-be1d87c8d67d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:44.385892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d44d8-6752-453c-9151-9ef7d4369c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:44.387024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d44d8-6752-453c-9151-9ef7d4369c8c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:44.387516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d44d8-6752-453c-9151-9ef7d4369c8c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:44.387976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d44d8-6752-453c-9151-9ef7d4369c8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:44.389525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d44d8-6752-453c-9151-9ef7d4369c8c None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=5764822dbac4490fa04caf5f1898050d, user.id=88008d0c94b14b0ba660fe163dc11662) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:44.390073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d44d8-6752-453c-9151-9ef7d4369c8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:44.392732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d44d8-6752-453c-9151-9ef7d4369c8c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:44.402043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 521/1059] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:44 2022] GET /identity/v3/role_assignments?user.id=88008d0c94b14b0ba660fe163dc11662&scope.project.id=5764822dbac4490fa04caf5f1898050d&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:45.148013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d412cb-3d07-43af-b498-6445d9e5cf3d None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:45.148955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d412cb-3d07-43af-b498-6445d9e5cf3d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:45.149501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d412cb-3d07-43af-b498-6445d9e5cf3d None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:45.152765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 539/1060] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:45.158881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e405203-b6f2-42df-b8ab-9bb25fa94451 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:45.159691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e405203-b6f2-42df-b8ab-9bb25fa94451 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:45.159691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e405203-b6f2-42df-b8ab-9bb25fa94451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:45.178665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-1e405203-b6f2-42df-b8ab-9bb25fa94451 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:45.179944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-1e405203-b6f2-42df-b8ab-9bb25fa94451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:45.269542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 522/1061] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:45.295008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d412cb-3d07-43af-b498-6445d9e5cf3d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:45.442460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d412cb-3d07-43af-b498-6445d9e5cf3d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:45.508921 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8cad5-cba4-4e91-81e6-7e3049dac61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:45.510222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8cad5-cba4-4e91-81e6-7e3049dac61d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:45.510651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8cad5-cba4-4e91-81e6-7e3049dac61d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:45.511077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8cad5-cba4-4e91-81e6-7e3049dac61d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:45.530950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8cad5-cba4-4e91-81e6-7e3049dac61d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-9ee8cad5-cba4-4e91-81e6-7e3049dac61d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:45.545438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:45.546646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:45.554141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 540/1062] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 4 14:39:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:45.559365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e405203-b6f2-42df-b8ab-9bb25fa94451 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:45.730050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e405203-b6f2-42df-b8ab-9bb25fa94451 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:45.737360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec8af3-ede5-471f-8da9-1db223b75a1c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:45.737762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec8af3-ede5-471f-8da9-1db223b75a1c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:45.737910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec8af3-ede5-471f-8da9-1db223b75a1c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:45.739775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 541/1063] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:45.745788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36bcee-864a-4132-885b-8d756d6bce9b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:45.746166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36bcee-864a-4132-885b-8d756d6bce9b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:45.746379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36bcee-864a-4132-885b-8d756d6bce9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:45.768044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1a36bcee-864a-4132-885b-8d756d6bce9b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:45.769336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1a36bcee-864a-4132-885b-8d756d6bce9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:45.791923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:45.793448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:45.794003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:45.794533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:45.796030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:45.796498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:45.799380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:45.821043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 523/1064] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 4 14:39:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:45.823335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:45.890323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 542/1065] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:45.914959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36bcee-864a-4132-885b-8d756d6bce9b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:45.985463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7301-0a5b-4344-a698-f115d1fc1d91 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:46.035994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:46.037419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.038059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.038597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.058560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:46.061334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:46.066796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36bcee-864a-4132-885b-8d756d6bce9b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:46.090510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:46.092015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:46.093866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:46.093866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:46.093866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:46.093866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:46.093866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 524/1066] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:46.098949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:46.119835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:46.121007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.121505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.121969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.225818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:46.227303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:46.244960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815435b-3e2a-4e18-9a98-b7bf169dcb5e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:46.296424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:46.297261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.297583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.297993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.299497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:46.299497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:46.300671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:46.316074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 525/1067] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:46.322536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:46.381128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 543/1068] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 04 14:39:46.387612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:46.475998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302110bf-c04f-4943-9d3e-92159c31a95c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:46.531098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:46.532106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.532367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.532670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.542141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:46.543770 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:46.544638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241090af-b3a5-4d19-a803-df17295c3b7e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:46.560962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:46.562798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:46.564338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 526/1069] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:39:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:46.570750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:46.614322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:46.618855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.618855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.618855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.636128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:46.638620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:46.663431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:46.664630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:46.665764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:46.665764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:46.665764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:46.665764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:46.670214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 544/1070] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:46.671130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:46.717205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aed886-dcbd-445f-9af4-1c91c5238d34 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:46.772401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:46.774063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.774825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.775574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.777416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:46.778274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:46.781093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:46.812830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf401e-e9bc-418b-803b-8b59f09ee145 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:46.813242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 527/1071] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:39:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:46.821134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:46.870372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16c795-6e69-4a40-84a2-bd3f16078b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:46.871494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16c795-6e69-4a40-84a2-bd3f16078b9d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.871931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16c795-6e69-4a40-84a2-bd3f16078b9d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.872417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16c795-6e69-4a40-84a2-bd3f16078b9d None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.873873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16c795-6e69-4a40-84a2-bd3f16078b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:46.874441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16c795-6e69-4a40-84a2-bd3f16078b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:46.877101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16c795-6e69-4a40-84a2-bd3f16078b9d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:46.898571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 545/1072] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:46.913160 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110130e-2a70-4e57-b4d6-1b8a598c5154 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.913457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110130e-2a70-4e57-b4d6-1b8a598c5154 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.913734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110130e-2a70-4e57-b4d6-1b8a598c5154 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.917035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 546/1073] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:46.921623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2599b-2746-453c-b4b7-f1efc81d4fc3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:46.922136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2599b-2746-453c-b4b7-f1efc81d4fc3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:46.922136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2599b-2746-453c-b4b7-f1efc81d4fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:46.959316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a9e2599b-2746-453c-b4b7-f1efc81d4fc3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:46.961200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a9e2599b-2746-453c-b4b7-f1efc81d4fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:46.977640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8af481-e27e-4487-9665-a446547a8cd0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.032185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71717339-4048-4cc9-a461-499819bec341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.033064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717339-4048-4cc9-a461-499819bec341 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.033396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717339-4048-4cc9-a461-499819bec341 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.033707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717339-4048-4cc9-a461-499819bec341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.046363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71717339-4048-4cc9-a461-499819bec341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.047945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71717339-4048-4cc9-a461-499819bec341 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.055267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 547/1074] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-71717339-4048-4cc9-a461-499819bec341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:47.070861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:47.072646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:47.074734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:47.074734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:47.074734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:47.074734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:47.074734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 528/1075] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:47.080662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2599b-2746-453c-b4b7-f1efc81d4fc3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.080920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71717339-4048-4cc9-a461-499819bec341 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.207064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2599b-2746-453c-b4b7-f1efc81d4fc3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.232512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71717339-4048-4cc9-a461-499819bec341 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.248427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.249440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.249915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.250421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.251952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:47.252550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.255190 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.274284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 548/1076] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:47.282998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.283638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.284054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.284494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.284894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.298159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:47.306781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:47.307975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:47.309001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 529/1077] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:47.316901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.418309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1498e2-53ca-42c9-a8ac-2213faa93a8b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.437798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c0e4c-7a53-461e-ab42-67ce6ac509bc None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.454036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.454812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.455223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.455497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.456414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:47.456842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.458400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.467402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 530/1078] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:47.475504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.487878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.488889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.489173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.489445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.497894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.499377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:47.516244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:47.517419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:47.518526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:47.521262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 549/1079] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:47.524759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.607158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920592c8-df96-4df4-96a6-c16cb1ed2382 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.647078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.648194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.648479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.648751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] PATH_INFO: `/v3/users/88008d0c94b14b0ba660fe163dc11662` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.650172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] RBAC: Authorizing `identity:get_user(user_id=88008d0c94b14b0ba660fe163dc11662)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.651716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.653539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 531/1080] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:39:47 2022] GET /identity/v3/users/88008d0c94b14b0ba660fe163dc11662 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:47.660504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.672902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f5949-9890-4f1b-be77-1e19d75697dd None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.730199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.731321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.731848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.732327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.733893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:47.734365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.737216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.759485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 550/1081] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:47.768721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.781717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdac32-b982-4612-8e93-dfa730f2e7d2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.824333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.825287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.825574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.825863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.827287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] RBAC: Authorizing `identity:get_project(project_id=5764822dbac4490fa04caf5f1898050d)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.828902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.830821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 532/1082] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:39:47 2022] GET /identity/v3/projects/5764822dbac4490fa04caf5f1898050d => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:47.837889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:47.889600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb9543-129d-4609-9911-4aa61ce1b862 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:47.929606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8f124-234e-45b7-a4b3-e68d24ec69c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:47.930597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8f124-234e-45b7-a4b3-e68d24ec69c6 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:47.931017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8f124-234e-45b7-a4b3-e68d24ec69c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:47.931451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8f124-234e-45b7-a4b3-e68d24ec69c6 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/753f2914e5bc4e298b99529e230d9ee3/roles/a5413484a9f84016bf6ffe3510bb7f75` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:47.934314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8f124-234e-45b7-a4b3-e68d24ec69c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=753f2914e5bc4e298b99529e230d9ee3, role_id=a5413484a9f84016bf6ffe3510bb7f75)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:47.936101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8f124-234e-45b7-a4b3-e68d24ec69c6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:47.952367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 551/1083] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:39:47 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/753f2914e5bc4e298b99529e230d9ee3/roles/a5413484a9f84016bf6ffe3510bb7f75 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:39:47.961957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a41ff-a98c-4a91-9cd5-37bff1f6fc3e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:48.004678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2079a3-23c0-46a6-8e48-d0e8868e57fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:48.005620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2079a3-23c0-46a6-8e48-d0e8868e57fa None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:48.005898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2079a3-23c0-46a6-8e48-d0e8868e57fa None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:48.006174 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2079a3-23c0-46a6-8e48-d0e8868e57fa None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d/users/88008d0c94b14b0ba660fe163dc11662/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:48.008384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2079a3-23c0-46a6-8e48-d0e8868e57fa None admin] RBAC: Authorizing `identity:create_grant(project_id=5764822dbac4490fa04caf5f1898050d, user_id=88008d0c94b14b0ba660fe163dc11662, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:48.010033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2079a3-23c0-46a6-8e48-d0e8868e57fa None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:48.026373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 533/1084] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:39:47 2022] PUT /identity/v3/projects/5764822dbac4490fa04caf5f1898050d/users/88008d0c94b14b0ba660fe163dc11662/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:39:50.380710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8bde3-661d-44b4-91ad-8ab3aac1c71e None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.381316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8bde3-661d-44b4-91ad-8ab3aac1c71e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.381316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8bde3-661d-44b4-91ad-8ab3aac1c71e None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.382802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 552/1085] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:50.389350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85c0d-2a11-46a5-9a17-e374eada0cef None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.389957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85c0d-2a11-46a5-9a17-e374eada0cef None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.390086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85c0d-2a11-46a5-9a17-e374eada0cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.418047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-87c85c0d-2a11-46a5-9a17-e374eada0cef None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:50.418938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a1a69-9f56-44a6-af47-9ed5c3b51d58 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.419225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a1a69-9f56-44a6-af47-9ed5c3b51d58 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.419497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a1a69-9f56-44a6-af47-9ed5c3b51d58 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.420044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-87c85c0d-2a11-46a5-9a17-e374eada0cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:50.420881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 553/1086] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:39:50.426479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacc51a-06dd-4f7d-ada7-65432b3e3396 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.426756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacc51a-06dd-4f7d-ada7-65432b3e3396 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.427033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacc51a-06dd-4f7d-ada7-65432b3e3396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.445607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8dacc51a-06dd-4f7d-ada7-65432b3e3396 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:50.446941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8dacc51a-06dd-4f7d-ada7-65432b3e3396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:50.520626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 554/1087] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:50.526117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 534/1088] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:39:50.544981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacc51a-06dd-4f7d-ada7-65432b3e3396 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:50.557709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c85c0d-2a11-46a5-9a17-e374eada0cef None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:50.671881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c85c0d-2a11-46a5-9a17-e374eada0cef None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:50.690827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacc51a-06dd-4f7d-ada7-65432b3e3396 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:50.713642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:50.714459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.714878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.715200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.730732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:50.742986 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:50.744198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.744627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.745080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:50.745320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:50.746661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:50.747686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 535/1089] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:39:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:50.758660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:50.763553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:39:50.778526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:50.779587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 04 14:39:50.780447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:50.781798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 555/1090] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 4 14:39:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:50.787736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:50.902598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd01ed4-001b-4b31-8f89-3fda03173322 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:50.918030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29d456-7e8b-4a32-8af7-a8cbe5b8d80b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:50.944097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:50.944788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.945218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.945554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.946424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:50.946795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:50.948293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:50.962072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 536/1091] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:39:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:50.966220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:50.969109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:50.969343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:50.969645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:50.970145 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:50.971081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:50.971396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:50.972818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:50.982097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 556/1092] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 4 14:39:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:50.989674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.106162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acf6fb-0238-4178-b838-1d4e4751e50a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.110746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9e9df-99ef-4d35-a404-629123f2fb29 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.143868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.144710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.145065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.145363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.159981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.160899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.161195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.161461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] PATH_INFO: `/v3/users/88008d0c94b14b0ba660fe163dc11662` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.162670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.162830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] RBAC: Authorizing `identity:get_user(user_id=88008d0c94b14b0ba660fe163dc11662)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.164223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.164457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.166081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 537/1093] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:39:50 2022] GET /identity/v3/users/88008d0c94b14b0ba660fe163dc11662 => generated 285 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:51.172780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:51.184781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:51.186055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:51.187179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:51.187179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:51.187179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:51.187179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:51.187179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 557/1094] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:39:51.192563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.307609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949d185-7d6f-495e-93ca-1672253b6f62 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.318346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b17995-0aba-44b8-9fd7-77936bef6dd0 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.347231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.348073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.348411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.348742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.361104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.361915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.362271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.362736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.363939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:51.364405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.366548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.389586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 558/1095] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:51.390827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.447344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.449625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.503624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69d8a-a471-43e3-a042-ea12f4f9966a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.552514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.554075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.554564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.555056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.570273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.573138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.593752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 538/1096] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:51.597590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:39:51.602758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:51.603896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 04 14:39:51.605213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:39:51.606034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 559/1097] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 4 14:39:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:51.618729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.706360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7499-871e-4d60-8a4f-7c9ddfcfbad4 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.755418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ba6a3-d288-4892-b5e5-f3b90cc3a3f7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.759522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.760953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.761237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.761509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.762739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] RBAC: Authorizing `identity:get_project(project_id=5764822dbac4490fa04caf5f1898050d)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.764169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.765931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 539/1098] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:39:51 2022] GET /identity/v3/projects/5764822dbac4490fa04caf5f1898050d => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:51.772766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.801605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.802433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.802873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.803176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.804445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:51.804902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.806758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.839070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 560/1099] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 4 14:39:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:51.840449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:51.893106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa70202-6af4-4548-b477-c34206511a3d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:51.945513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:51.946536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:51.946979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:51.947307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:51.948722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=5764822dbac4490fa04caf5f1898050d, user.id=88008d0c94b14b0ba660fe163dc11662) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:51.949252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:51.951590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:51.972866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 540/1100] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:51 2022] GET /identity/v3/role_assignments?user.id=88008d0c94b14b0ba660fe163dc11662&scope.project.id=5764822dbac4490fa04caf5f1898050d&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:39:51.983191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537066b-08e4-4991-9297-9adb439888ef None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:52.029354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:52.030072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:52.030354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:52.030628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:52.118946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:52.120496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:52.252912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 561/1101] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:52.259942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:52.390308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6db5b2-21d1-44e8-adfe-e3a7c06770b2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:52.432748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:52.433595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:52.433872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:52.434143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:52.445454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:52.446906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:39:52.465987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:52.468659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:52.471010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:39:52.471010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:39:52.471010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:39:52.471010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:52.471010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 541/1102] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:52.473315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:52.613590 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7a80f-c40b-4be8-85be-0f5e23b8e150 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:52.653651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:52.654242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:52.654519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:52.654799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:52.655521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:52.655914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:52.657335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:52.670035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 562/1103] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:52.678448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:52.788051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7f3c-3e6c-47fc-8547-c2d58816f903 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:52.825380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:52.826384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:52.826768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:52.827161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:52.836432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:52.838041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:39:52.854348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:39:52.855652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:39:52.857232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 542/1104] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:39:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:39:52.862427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:52.967733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb71605-2cd1-4da7-8d0d-7de79fe2a33d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:53.008451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f724c-6449-43bb-9c05-cc716cec192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:53.009169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f724c-6449-43bb-9c05-cc716cec192b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:53.009449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f724c-6449-43bb-9c05-cc716cec192b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:53.009727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f724c-6449-43bb-9c05-cc716cec192b None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:53.010528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f724c-6449-43bb-9c05-cc716cec192b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:53.010843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f724c-6449-43bb-9c05-cc716cec192b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:53.012373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f724c-6449-43bb-9c05-cc716cec192b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:53.027731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 563/1105] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:39:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:53.035487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:53.145831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cca0e2-a7ba-4a60-ab7e-749dade1a7ea None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:53.188728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe09c9-ab65-4303-a806-e9392f5d325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:53.189424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe09c9-ab65-4303-a806-e9392f5d325e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:53.189716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe09c9-ab65-4303-a806-e9392f5d325e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:53.190048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe09c9-ab65-4303-a806-e9392f5d325e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:53.190927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe09c9-ab65-4303-a806-e9392f5d325e None admin] RBAC: Adding query filter params (role.id=a5413484a9f84016bf6ffe3510bb7f75, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=753f2914e5bc4e298b99529e230d9ee3) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:53.191204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe09c9-ab65-4303-a806-e9392f5d325e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:53.192776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe09c9-ab65-4303-a806-e9392f5d325e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:53.199759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 543/1106] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:53 2022] GET /identity/v3/role_assignments?user.id=753f2914e5bc4e298b99529e230d9ee3&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=a5413484a9f84016bf6ffe3510bb7f75 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:54.432987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc48afb-09f3-42b6-94f1-d5e12896cdd0 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:54.433540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc48afb-09f3-42b6-94f1-d5e12896cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:54.433540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc48afb-09f3-42b6-94f1-d5e12896cdd0 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:54.434896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 564/1107] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:54.440936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921a996-c86a-4beb-8d75-235aac6fc898 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:54.441491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921a996-c86a-4beb-8d75-235aac6fc898 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:54.441491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921a996-c86a-4beb-8d75-235aac6fc898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:54.459743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a921a996-c86a-4beb-8d75-235aac6fc898 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:54.461129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a921a996-c86a-4beb-8d75-235aac6fc898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:54.530614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 544/1108] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:54.568741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc48afb-09f3-42b6-94f1-d5e12896cdd0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:54.702846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc48afb-09f3-42b6-94f1-d5e12896cdd0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:54.739630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525cfcb-f5ad-4c71-85c7-2d1c606d67b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:54.740417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525cfcb-f5ad-4c71-85c7-2d1c606d67b4 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:54.740753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525cfcb-f5ad-4c71-85c7-2d1c606d67b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:54.741063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525cfcb-f5ad-4c71-85c7-2d1c606d67b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:54.742295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525cfcb-f5ad-4c71-85c7-2d1c606d67b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:54.743735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525cfcb-f5ad-4c71-85c7-2d1c606d67b4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:54.745424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 565/1109] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:39:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:54.751527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921a996-c86a-4beb-8d75-235aac6fc898 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:54.880916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921a996-c86a-4beb-8d75-235aac6fc898 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:54.917312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c2da5-4e6d-40f3-8431-7cc9afd103d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:54.918011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2da5-4e6d-40f3-8431-7cc9afd103d8 None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:54.918291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2da5-4e6d-40f3-8431-7cc9afd103d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:54.918565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2da5-4e6d-40f3-8431-7cc9afd103d8 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:54.919294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c2da5-4e6d-40f3-8431-7cc9afd103d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:54.920882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c2da5-4e6d-40f3-8431-7cc9afd103d8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:54.943517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 545/1110] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:39:54 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:39:57.353118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b6cdc-8f4b-463b-af84-ea1439f8ed9b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:57.353705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b6cdc-8f4b-463b-af84-ea1439f8ed9b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:57.353705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b6cdc-8f4b-463b-af84-ea1439f8ed9b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:57.355012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 566/1111] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:39:57.361001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c8fa3-bd3c-41fe-ac28-fe95bb2b46c8 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:57.361432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c8fa3-bd3c-41fe-ac28-fe95bb2b46c8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:57.361520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c8fa3-bd3c-41fe-ac28-fe95bb2b46c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:57.379383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d52c8fa3-bd3c-41fe-ac28-fe95bb2b46c8 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:39:57.380737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d52c8fa3-bd3c-41fe-ac28-fe95bb2b46c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:39:57.454197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 546/1112] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:39:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:39:57.480580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b6cdc-8f4b-463b-af84-ea1439f8ed9b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:57.598182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b6cdc-8f4b-463b-af84-ea1439f8ed9b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:57.633690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:57.634553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:57.634828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:57.635099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] PATH_INFO: `/v3/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:57.636652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] RBAC: Authorizing `identity:get_role(role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:57.638166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:57.639939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 567/1113] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:39:57 2022] GET /identity/v3/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:57.646556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c8fa3-bd3c-41fe-ac28-fe95bb2b46c8 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:57.754901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c8fa3-bd3c-41fe-ac28-fe95bb2b46c8 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:57.790677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:57.791568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:57.791839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:57.792106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] PATH_INFO: `/v3/users/4dd3f8de10f247338275c8a8ab29b04a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:57.806981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] RBAC: Authorizing `identity:get_user(user_id=4dd3f8de10f247338275c8a8ab29b04a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:57.808556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:57.810434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 547/1114] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:39:57 2022] GET /identity/v3/users/4dd3f8de10f247338275c8a8ab29b04a => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:57.816736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:57.926862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ecc60-aa1b-41a6-a7f5-215f4272ecab None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:57.968632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:57.969385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:57.969682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:57.970045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:58.040281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:58.041765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:58.141444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 568/1115] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:39:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:39:58.151172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:58.257121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02b3d4-dfeb-4a7a-95ad-38144395bb60 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:58.293708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafe4ac-5787-417b-8fbd-26f1fa8e7d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:58.294566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafe4ac-5787-417b-8fbd-26f1fa8e7d5a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:58.294846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafe4ac-5787-417b-8fbd-26f1fa8e7d5a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:58.295318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafe4ac-5787-417b-8fbd-26f1fa8e7d5a None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:58.296637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafe4ac-5787-417b-8fbd-26f1fa8e7d5a None admin] RBAC: Authorizing `identity:get_project(project_id=5764822dbac4490fa04caf5f1898050d)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:58.298825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafe4ac-5787-417b-8fbd-26f1fa8e7d5a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:58.300927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 548/1116] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:39:58 2022] GET /identity/v3/projects/5764822dbac4490fa04caf5f1898050d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:39:58.307280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:39:58.416549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f266be-5a48-4e9f-84b5-262859e11148 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:39:58.455040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d19bf3-f1bc-402e-b950-f2f587feb812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:39:58.455704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d19bf3-f1bc-402e-b950-f2f587feb812 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:39:58.456009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d19bf3-f1bc-402e-b950-f2f587feb812 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:39:58.456417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d19bf3-f1bc-402e-b950-f2f587feb812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:39:58.457424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d19bf3-f1bc-402e-b950-f2f587feb812 None admin] RBAC: Adding query filter params (role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=5764822dbac4490fa04caf5f1898050d, user.id=4dd3f8de10f247338275c8a8ab29b04a) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:39:58.457831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d19bf3-f1bc-402e-b950-f2f587feb812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:39:58.459499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d19bf3-f1bc-402e-b950-f2f587feb812 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:39:58.467340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 569/1117] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:39:58 2022] GET /identity/v3/role_assignments?user.id=4dd3f8de10f247338275c8a8ab29b04a&scope.project.id=5764822dbac4490fa04caf5f1898050d&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:00.892454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629eaab-2673-4d02-9e8c-6ac18d121670 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:00.893063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629eaab-2673-4d02-9e8c-6ac18d121670 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:00.893165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629eaab-2673-4d02-9e8c-6ac18d121670 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:00.894563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 549/1118] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:00.901368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710df6c-e1df-4adc-acdb-2db1b317c358 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:00.901867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710df6c-e1df-4adc-acdb-2db1b317c358 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:00.901867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710df6c-e1df-4adc-acdb-2db1b317c358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:00.926398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-9710df6c-e1df-4adc-acdb-2db1b317c358 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:00.928028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9710df6c-e1df-4adc-acdb-2db1b317c358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:01.009724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 570/1119] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:01.033104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629eaab-2673-4d02-9e8c-6ac18d121670 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:01.145757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629eaab-2673-4d02-9e8c-6ac18d121670 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:01.184053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:01.185081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:01.185457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:01.185719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] PATH_INFO: `/v3/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:01.187249 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] RBAC: Authorizing `identity:get_role(role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:01.188666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:01.190350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 550/1120] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:01 2022] GET /identity/v3/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:01.198074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710df6c-e1df-4adc-acdb-2db1b317c358 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:01.327729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710df6c-e1df-4adc-acdb-2db1b317c358 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:01.366750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:01.367672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:01.367956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:01.368231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] PATH_INFO: `/v3/users/4dd3f8de10f247338275c8a8ab29b04a` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:01.369454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] RBAC: Authorizing `identity:get_user(user_id=4dd3f8de10f247338275c8a8ab29b04a)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:01.370904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:01.372648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 571/1121] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:01 2022] GET /identity/v3/users/4dd3f8de10f247338275c8a8ab29b04a => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:01.379338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:01.485567 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1744d9-c217-4430-936b-784cc50ff9e7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:01.521751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179ea4-c992-40fb-8673-c20ae33437ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:01.522746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179ea4-c992-40fb-8673-c20ae33437ff None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:01.523054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179ea4-c992-40fb-8673-c20ae33437ff None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:01.523349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179ea4-c992-40fb-8673-c20ae33437ff None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:01.524717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179ea4-c992-40fb-8673-c20ae33437ff None admin] RBAC: Authorizing `identity:get_project(project_id=5764822dbac4490fa04caf5f1898050d)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:01.526369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179ea4-c992-40fb-8673-c20ae33437ff None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:01.528333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 551/1122] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:01 2022] GET /identity/v3/projects/5764822dbac4490fa04caf5f1898050d => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:01.535228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:01.651099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b1966-4c24-4730-9bb0-b2bec63f08bf None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:01.690468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2c454-0798-4a74-a839-ff03a10debd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:01.691522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2c454-0798-4a74-a839-ff03a10debd4 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:01.691926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2c454-0798-4a74-a839-ff03a10debd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:01.692244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2c454-0798-4a74-a839-ff03a10debd4 None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d/users/4dd3f8de10f247338275c8a8ab29b04a/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:01.695239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2c454-0798-4a74-a839-ff03a10debd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5764822dbac4490fa04caf5f1898050d, user_id=4dd3f8de10f247338275c8a8ab29b04a, role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:01.696831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2c454-0798-4a74-a839-ff03a10debd4 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:01.711928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 572/1123] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:40:01 2022] PUT /identity/v3/projects/5764822dbac4490fa04caf5f1898050d/users/4dd3f8de10f247338275c8a8ab29b04a/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:40:04.132279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666df72-b3a9-4548-b10a-469d36a5bdfa None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:04.132894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666df72-b3a9-4548-b10a-469d36a5bdfa None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:04.132894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666df72-b3a9-4548-b10a-469d36a5bdfa None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:04.134250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 552/1124] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:04.140030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ed68-bd40-47fb-8ae1-e3adce4d690f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:04.140917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ed68-bd40-47fb-8ae1-e3adce4d690f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:04.140917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24ed68-bd40-47fb-8ae1-e3adce4d690f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:04.160753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-fa24ed68-bd40-47fb-8ae1-e3adce4d690f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:04.162436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fa24ed68-bd40-47fb-8ae1-e3adce4d690f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:04.241436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 573/1125] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:40:04.267576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666df72-b3a9-4548-b10a-469d36a5bdfa None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:04.378733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666df72-b3a9-4548-b10a-469d36a5bdfa None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:04.417091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:04.417900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:04.418291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:04.418563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] PATH_INFO: `/v3/roles/92eb4c2f37474c338c51f9f847596cf6` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:04.420291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] RBAC: Authorizing `identity:get_role(role_id=92eb4c2f37474c338c51f9f847596cf6)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:04.421747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:04.423320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 553/1126] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:04 2022] GET /identity/v3/roles/92eb4c2f37474c338c51f9f847596cf6 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:40:04.429590 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24ed68-bd40-47fb-8ae1-e3adce4d690f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:04.537361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24ed68-bd40-47fb-8ae1-e3adce4d690f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:04.573559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:04.574484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:04.574807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:04.575113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] PATH_INFO: `/v3/users/4dd3f8de10f247338275c8a8ab29b04a` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:04.576369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] RBAC: Authorizing `identity:get_user(user_id=4dd3f8de10f247338275c8a8ab29b04a)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:04.577850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:04.579647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 574/1127] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:04 2022] GET /identity/v3/users/4dd3f8de10f247338275c8a8ab29b04a => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:04.586415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:04.695942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229e15-2207-4c62-9757-9edbb4a27290 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:04.731296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:04.731939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:04.732203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:04.732485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:04.802371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:04.803844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:04.909193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 554/1128] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:40:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:40:04.915735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:05.029376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36f551-10d7-4fbe-82f6-92d6b7f8c7d2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:05.065578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10929f83-6312-4ad2-82c4-4dc47d747488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:05.066451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929f83-6312-4ad2-82c4-4dc47d747488 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:05.066725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929f83-6312-4ad2-82c4-4dc47d747488 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:05.066985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929f83-6312-4ad2-82c4-4dc47d747488 None admin] PATH_INFO: `/v3/projects/5764822dbac4490fa04caf5f1898050d` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:05.068190 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10929f83-6312-4ad2-82c4-4dc47d747488 None admin] RBAC: Authorizing `identity:get_project(project_id=5764822dbac4490fa04caf5f1898050d)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:05.069696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10929f83-6312-4ad2-82c4-4dc47d747488 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:05.071674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 575/1129] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:04 2022] GET /identity/v3/projects/5764822dbac4490fa04caf5f1898050d => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:05.078222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:05.190169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff1e52-a2f0-46e7-9baa-a23124880629 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:05.238338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b41fd-0767-4bb2-9158-2dcf76ad5413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:05.239076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b41fd-0767-4bb2-9158-2dcf76ad5413 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:05.239292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b41fd-0767-4bb2-9158-2dcf76ad5413 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:05.239630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b41fd-0767-4bb2-9158-2dcf76ad5413 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:05.240586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b41fd-0767-4bb2-9158-2dcf76ad5413 None admin] RBAC: Adding query filter params (role.id=92eb4c2f37474c338c51f9f847596cf6, scope.project.id=5764822dbac4490fa04caf5f1898050d, user.id=4dd3f8de10f247338275c8a8ab29b04a) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:05.240922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b41fd-0767-4bb2-9158-2dcf76ad5413 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:05.242525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b41fd-0767-4bb2-9158-2dcf76ad5413 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:05.249707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 555/1130] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:40:05 2022] GET /identity/v3/role_assignments?user.id=4dd3f8de10f247338275c8a8ab29b04a&scope.project.id=5764822dbac4490fa04caf5f1898050d&role.id=92eb4c2f37474c338c51f9f847596cf6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:07.687198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde58ae7-5cf0-4eeb-b3a2-50844787c1e8 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:07.687799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde58ae7-5cf0-4eeb-b3a2-50844787c1e8 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:07.687799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde58ae7-5cf0-4eeb-b3a2-50844787c1e8 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:07.689101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 576/1131] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:07.694803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0afb7-c2ef-45f5-9b5f-4eb1855d87dc None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:07.695273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0afb7-c2ef-45f5-9b5f-4eb1855d87dc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:07.695454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0afb7-c2ef-45f5-9b5f-4eb1855d87dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:07.713585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a6d0afb7-c2ef-45f5-9b5f-4eb1855d87dc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:07.714770 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a6d0afb7-c2ef-45f5-9b5f-4eb1855d87dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:07.782349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 556/1132] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:07.805540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde58ae7-5cf0-4eeb-b3a2-50844787c1e8 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:07.916415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde58ae7-5cf0-4eeb-b3a2-50844787c1e8 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:07.950682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88487a5-b483-495e-83c6-aa30f98af8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:07.951391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88487a5-b483-495e-83c6-aa30f98af8bc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:07.951665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88487a5-b483-495e-83c6-aa30f98af8bc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:07.951934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88487a5-b483-495e-83c6-aa30f98af8bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:07.953178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88487a5-b483-495e-83c6-aa30f98af8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:07.954593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88487a5-b483-495e-83c6-aa30f98af8bc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:07.956320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 577/1133] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:40:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:07.962637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0afb7-c2ef-45f5-9b5f-4eb1855d87dc None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:08.062093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0afb7-c2ef-45f5-9b5f-4eb1855d87dc None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:08.096523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024a87d-7da9-45cc-a827-34600147131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:08.097219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024a87d-7da9-45cc-a827-34600147131d None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:08.097513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024a87d-7da9-45cc-a827-34600147131d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:08.097787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024a87d-7da9-45cc-a827-34600147131d None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:08.098496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024a87d-7da9-45cc-a827-34600147131d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:08.099961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024a87d-7da9-45cc-a827-34600147131d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:08.126668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 557/1134] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:40:07 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:40:10.381060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fe5db-f33b-47f0-851b-e79f09f21873 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:10.381613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fe5db-f33b-47f0-851b-e79f09f21873 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:10.381613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fe5db-f33b-47f0-851b-e79f09f21873 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:10.382978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 578/1135] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:10.389146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35552ca-e9f0-45ec-aecf-245511edace6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:10.389601 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35552ca-e9f0-45ec-aecf-245511edace6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:10.389708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35552ca-e9f0-45ec-aecf-245511edace6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:10.408157 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e35552ca-e9f0-45ec-aecf-245511edace6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:10.409411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e35552ca-e9f0-45ec-aecf-245511edace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:10.480602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 558/1136] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:10.505528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fe5db-f33b-47f0-851b-e79f09f21873 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:10.613618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fe5db-f33b-47f0-851b-e79f09f21873 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:10.649528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7685c79-7724-4245-b6ab-93fb04cfa432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:10.650347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685c79-7724-4245-b6ab-93fb04cfa432 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:10.650625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685c79-7724-4245-b6ab-93fb04cfa432 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:10.650904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7685c79-7724-4245-b6ab-93fb04cfa432 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:10.652150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7685c79-7724-4245-b6ab-93fb04cfa432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:10.653637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7685c79-7724-4245-b6ab-93fb04cfa432 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:10.655287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 579/1137] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:40:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:10.661763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35552ca-e9f0-45ec-aecf-245511edace6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:10.766654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35552ca-e9f0-45ec-aecf-245511edace6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:10.804659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cadd1-1967-4025-b40e-2eaad987cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:10.805398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cadd1-1967-4025-b40e-2eaad987cf4a None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:10.805676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cadd1-1967-4025-b40e-2eaad987cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:10.805952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cadd1-1967-4025-b40e-2eaad987cf4a None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:10.806689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cadd1-1967-4025-b40e-2eaad987cf4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:10.808169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cadd1-1967-4025-b40e-2eaad987cf4a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:10.832476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 559/1138] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:40:10 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:40:13.126616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000aa0a-3be4-4a4a-98a9-b31b718397cb None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:13.127233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000aa0a-3be4-4a4a-98a9-b31b718397cb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:13.127233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000aa0a-3be4-4a4a-98a9-b31b718397cb None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:13.128488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 580/1139] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:13.134263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709e120-d1e9-4d36-ad67-c17ace5e7238 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:13.134717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709e120-d1e9-4d36-ad67-c17ace5e7238 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:13.134824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709e120-d1e9-4d36-ad67-c17ace5e7238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:13.152422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7709e120-d1e9-4d36-ad67-c17ace5e7238 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:13.153654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7709e120-d1e9-4d36-ad67-c17ace5e7238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:13.229309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 560/1140] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:13.252993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000aa0a-3be4-4a4a-98a9-b31b718397cb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:13.368226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000aa0a-3be4-4a4a-98a9-b31b718397cb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:13.403380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:13.404064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:13.404371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:13.404633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:13.415357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:40:13.423048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:40:13.424197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:40:13.425416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 581/1141] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:40:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:40:13.430717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709e120-d1e9-4d36-ad67-c17ace5e7238 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:13.533581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709e120-d1e9-4d36-ad67-c17ace5e7238 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:13.568574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:13.569274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:13.569556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:13.569828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:13.570576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:13.570879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:13.572253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:13.580774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 561/1142] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:40:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:13.586980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:13.694753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec026975-e02d-433e-93ba-bab065454375 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:13.736785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:13.737806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:13.738138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:13.738482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] PATH_INFO: `/v3/users/04f50061f5be42ec8644aeca8975f17b` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:13.753619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] RBAC: Authorizing `identity:get_user(user_id=04f50061f5be42ec8644aeca8975f17b)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:13.755245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:13.757098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 582/1143] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:13 2022] GET /identity/v3/users/04f50061f5be42ec8644aeca8975f17b => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:13.763166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:13.873589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c256a-a429-41a8-a19a-699c170d45a7 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:13.908632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:13.909425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:13.909684 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:13.909974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:13.976726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:13.978116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:14.077783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 562/1144] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:40:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:40:14.085249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:14.190648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1138548-e291-4591-aa3a-16f55daddc89 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:14.224336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00b67c-0c5f-4f9b-bf28-2845bf946119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:14.225292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b67c-0c5f-4f9b-bf28-2845bf946119 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:14.225566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b67c-0c5f-4f9b-bf28-2845bf946119 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:14.225838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b67c-0c5f-4f9b-bf28-2845bf946119 None admin] PATH_INFO: `/v3/projects/a26f8eca2e884aa597155b8c34daedec` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:14.227114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00b67c-0c5f-4f9b-bf28-2845bf946119 None admin] RBAC: Authorizing `identity:get_project(project_id=a26f8eca2e884aa597155b8c34daedec)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:14.228479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00b67c-0c5f-4f9b-bf28-2845bf946119 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:14.230224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 583/1145] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:14 2022] GET /identity/v3/projects/a26f8eca2e884aa597155b8c34daedec => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:14.236713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:14.341803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424f48e-e578-4c19-8273-a433e237913a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:14.378649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4730f-8a35-420c-995c-48af4b9256c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:14.379291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4730f-8a35-420c-995c-48af4b9256c3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:14.379568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4730f-8a35-420c-995c-48af4b9256c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:14.379836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4730f-8a35-420c-995c-48af4b9256c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:14.380716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4730f-8a35-420c-995c-48af4b9256c3 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=a26f8eca2e884aa597155b8c34daedec, user.id=04f50061f5be42ec8644aeca8975f17b) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:14.381052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4730f-8a35-420c-995c-48af4b9256c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:14.382468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4730f-8a35-420c-995c-48af4b9256c3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:14.388060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 563/1146] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:40:14 2022] GET /identity/v3/role_assignments?user.id=04f50061f5be42ec8644aeca8975f17b&scope.project.id=a26f8eca2e884aa597155b8c34daedec&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:16.600242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ae297-f48d-438f-ba9b-cbcb0f26cf5b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:16.600786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ae297-f48d-438f-ba9b-cbcb0f26cf5b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:16.600786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ae297-f48d-438f-ba9b-cbcb0f26cf5b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:16.602068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 584/1147] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:16.607638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5d01-600b-4fed-9014-be9d59ac14e6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:16.608093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5d01-600b-4fed-9014-be9d59ac14e6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:16.608194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5d01-600b-4fed-9014-be9d59ac14e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:16.625247 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-9bdd5d01-600b-4fed-9014-be9d59ac14e6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:16.626433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9bdd5d01-600b-4fed-9014-be9d59ac14e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:16.692711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 564/1148] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:40:16.715847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ae297-f48d-438f-ba9b-cbcb0f26cf5b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:16.825468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ae297-f48d-438f-ba9b-cbcb0f26cf5b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:16.860413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:16.861155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:16.861434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:16.861676 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:16.872057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:40:16.879483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:40:16.880591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:40:16.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 585/1149] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:40:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:40:16.887127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5d01-600b-4fed-9014-be9d59ac14e6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:16.987033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5d01-600b-4fed-9014-be9d59ac14e6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:17.021301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:17.021959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:17.022228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:17.022496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:17.023265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:17.023568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:17.024980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:17.033112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 565/1150] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:40:17.039702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:17.141065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6c54d-b744-41cb-b348-68a425d641d7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:17.175205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:17.176083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:17.176356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:17.176624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] PATH_INFO: `/v3/users/04f50061f5be42ec8644aeca8975f17b` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:17.178000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] RBAC: Authorizing `identity:get_user(user_id=04f50061f5be42ec8644aeca8975f17b)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:17.179442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:17.181129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 586/1151] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:17 2022] GET /identity/v3/users/04f50061f5be42ec8644aeca8975f17b => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:17.187340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:17.292575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce48fa9-ec3b-4204-b347-b7624720b6a1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:17.328705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7b58-41f6-4e43-9e58-25b048434678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:17.329598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7b58-41f6-4e43-9e58-25b048434678 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:17.329872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7b58-41f6-4e43-9e58-25b048434678 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:17.330136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7b58-41f6-4e43-9e58-25b048434678 None admin] PATH_INFO: `/v3/projects/a26f8eca2e884aa597155b8c34daedec` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:17.331354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7b58-41f6-4e43-9e58-25b048434678 None admin] RBAC: Authorizing `identity:get_project(project_id=a26f8eca2e884aa597155b8c34daedec)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:17.332772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7b58-41f6-4e43-9e58-25b048434678 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:17.334560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 566/1152] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:17 2022] GET /identity/v3/projects/a26f8eca2e884aa597155b8c34daedec => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:17.341059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:17.445808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfb0bf-8d35-41bf-bffe-8b5e6fa1a61d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:17.480471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ce4f6-e5be-483b-839b-2b8ef0df378f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:17.481385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ce4f6-e5be-483b-839b-2b8ef0df378f None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:17.481663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ce4f6-e5be-483b-839b-2b8ef0df378f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:17.481938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ce4f6-e5be-483b-839b-2b8ef0df378f None admin] PATH_INFO: `/v3/projects/a26f8eca2e884aa597155b8c34daedec/users/04f50061f5be42ec8644aeca8975f17b/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:17.484018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ce4f6-e5be-483b-839b-2b8ef0df378f None admin] RBAC: Authorizing `identity:create_grant(project_id=a26f8eca2e884aa597155b8c34daedec, user_id=04f50061f5be42ec8644aeca8975f17b, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:17.485481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ce4f6-e5be-483b-839b-2b8ef0df378f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:17.500344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 587/1153] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:40:17 2022] PUT /identity/v3/projects/a26f8eca2e884aa597155b8c34daedec/users/04f50061f5be42ec8644aeca8975f17b/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:40:19.722992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7413b4-b4df-485b-9af2-1aaaf72a3023 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:19.723530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7413b4-b4df-485b-9af2-1aaaf72a3023 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:19.723530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7413b4-b4df-485b-9af2-1aaaf72a3023 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:19.725515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 567/1154] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:19.731145 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d5555-fe00-46e6-aa97-134c0952ede9 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:19.731640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d5555-fe00-46e6-aa97-134c0952ede9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:19.731640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d5555-fe00-46e6-aa97-134c0952ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:19.748510 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-3b6d5555-fe00-46e6-aa97-134c0952ede9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:19.749707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-3b6d5555-fe00-46e6-aa97-134c0952ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:19.816658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 588/1155] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:19.840188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7413b4-b4df-485b-9af2-1aaaf72a3023 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:19.949532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7413b4-b4df-485b-9af2-1aaaf72a3023 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:19.982964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:19.983679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:19.983956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:19.984229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:19.995211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:40:20.002737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:40:20.003826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:40:20.004954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 568/1156] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:40:20.010301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d5555-fe00-46e6-aa97-134c0952ede9 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:20.107970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d5555-fe00-46e6-aa97-134c0952ede9 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:20.142273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:20.142942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:20.143223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:20.143495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:20.144260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:20.144565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:20.146401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:20.154781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 589/1157] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:20.160879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:20.264687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8deeb9-460b-470b-b406-de336417c36a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:20.300112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:20.301019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:20.301297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:20.301578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] PATH_INFO: `/v3/users/04f50061f5be42ec8644aeca8975f17b` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:20.302934 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] RBAC: Authorizing `identity:get_user(user_id=04f50061f5be42ec8644aeca8975f17b)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:20.304359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:20.306031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 569/1158] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:20 2022] GET /identity/v3/users/04f50061f5be42ec8644aeca8975f17b => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:20.312046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:20.415496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa888b-b64e-4c52-bba7-78a0cbc28271 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:20.449284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:20.449944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:20.450224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:20.450501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:20.519232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:20.520610 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:20.620502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 590/1159] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:40:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:40:20.626838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:20.725733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e45c9a-c234-40be-931f-0256374ef4b0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:20.763720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405f738-cd89-41da-8185-17f1ea8b2582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:20.764612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405f738-cd89-41da-8185-17f1ea8b2582 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:20.764922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405f738-cd89-41da-8185-17f1ea8b2582 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:20.765201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405f738-cd89-41da-8185-17f1ea8b2582 None admin] PATH_INFO: `/v3/projects/a26f8eca2e884aa597155b8c34daedec` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:20.766517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405f738-cd89-41da-8185-17f1ea8b2582 None admin] RBAC: Authorizing `identity:get_project(project_id=a26f8eca2e884aa597155b8c34daedec)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:20.767992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405f738-cd89-41da-8185-17f1ea8b2582 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:20.769939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 570/1160] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:20 2022] GET /identity/v3/projects/a26f8eca2e884aa597155b8c34daedec => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:20.776873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:20.880155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace6f65-8011-4391-86ca-9b7136293413 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:20.916127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cdd3f-6f59-4f92-977f-1fdfa2b3b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:20.916785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cdd3f-6f59-4f92-977f-1fdfa2b3b156 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:20.917090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cdd3f-6f59-4f92-977f-1fdfa2b3b156 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:20.917363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cdd3f-6f59-4f92-977f-1fdfa2b3b156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:20.918198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cdd3f-6f59-4f92-977f-1fdfa2b3b156 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=a26f8eca2e884aa597155b8c34daedec, user.id=04f50061f5be42ec8644aeca8975f17b) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:20.918508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cdd3f-6f59-4f92-977f-1fdfa2b3b156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:20.919957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cdd3f-6f59-4f92-977f-1fdfa2b3b156 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:20.926095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 591/1161] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:40:20 2022] GET /identity/v3/role_assignments?user.id=04f50061f5be42ec8644aeca8975f17b&scope.project.id=a26f8eca2e884aa597155b8c34daedec&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:23.185927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5432841-74a2-4e1f-ba2b-7bf8f92bee63 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:23.186511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5432841-74a2-4e1f-ba2b-7bf8f92bee63 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:23.186511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5432841-74a2-4e1f-ba2b-7bf8f92bee63 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:23.187852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 571/1162] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:23.193655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07e67-faed-45b2-aa33-c042371cc7f2 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:23.194378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07e67-faed-45b2-aa33-c042371cc7f2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:23.194378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa07e67-faed-45b2-aa33-c042371cc7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:23.213786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6aa07e67-faed-45b2-aa33-c042371cc7f2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:23.214998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6aa07e67-faed-45b2-aa33-c042371cc7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:23.285920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 592/1163] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:23.313357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5432841-74a2-4e1f-ba2b-7bf8f92bee63 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:23.430964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5432841-74a2-4e1f-ba2b-7bf8f92bee63 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:23.487424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:23.488215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:23.488500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:23.488782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:23.559096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:23.560540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:23.700650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 572/1164] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:40:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:40:23.710252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07e67-faed-45b2-aa33-c042371cc7f2 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:23.818864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa07e67-faed-45b2-aa33-c042371cc7f2 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:23.861340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbf72e-4a06-430b-93b3-5a88155b1b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:23.862175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbf72e-4a06-430b-93b3-5a88155b1b3d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:23.862457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbf72e-4a06-430b-93b3-5a88155b1b3d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:23.862731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbf72e-4a06-430b-93b3-5a88155b1b3d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:23.871451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbf72e-4a06-430b-93b3-5a88155b1b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:23.873118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbf72e-4a06-430b-93b3-5a88155b1b3d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-ccfbf72e-4a06-430b-93b3-5a88155b1b3d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:40:23.889333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:40:23.890932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:40:23.892339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 593/1165] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:40:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:40:23.899034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:24.013379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c9fc6-c71c-4b48-8ff0-7d21fc917720 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:24.052700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0faaa9-00f6-4343-870f-4cef09a4250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:24.053414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0faaa9-00f6-4343-870f-4cef09a4250f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:24.053696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0faaa9-00f6-4343-870f-4cef09a4250f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:24.053973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0faaa9-00f6-4343-870f-4cef09a4250f None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:24.054704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0faaa9-00f6-4343-870f-4cef09a4250f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:24.055010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0faaa9-00f6-4343-870f-4cef09a4250f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:24.056411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0faaa9-00f6-4343-870f-4cef09a4250f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:24.064663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 573/1166] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 4 14:40:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:26.287447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7a32f-afad-48b5-8e77-18562da0b168 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:26.288008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7a32f-afad-48b5-8e77-18562da0b168 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:26.288008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7a32f-afad-48b5-8e77-18562da0b168 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:26.289647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 594/1167] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:26.295948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9a06-124d-49aa-bb1e-53e267b58f4a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:26.296386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9a06-124d-49aa-bb1e-53e267b58f4a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:26.296488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9a06-124d-49aa-bb1e-53e267b58f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:26.315439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a98d9a06-124d-49aa-bb1e-53e267b58f4a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:26.316637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a98d9a06-124d-49aa-bb1e-53e267b58f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:26.393951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 574/1168] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:26.417119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7a32f-afad-48b5-8e77-18562da0b168 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:26.532155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7a32f-afad-48b5-8e77-18562da0b168 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:26.577006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916603b6-89bb-4b46-aea4-727ca4b165a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:26.577596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916603b6-89bb-4b46-aea4-727ca4b165a2 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:26.577874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916603b6-89bb-4b46-aea4-727ca4b165a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:26.578140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916603b6-89bb-4b46-aea4-727ca4b165a2 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:26.578887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916603b6-89bb-4b46-aea4-727ca4b165a2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:26.580262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916603b6-89bb-4b46-aea4-727ca4b165a2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:26.600443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 595/1169] 10.222.0.32 () {60 vars in 1138 bytes} [Wed May 4 14:40:26 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:40:28.899008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3f59f-8e36-42d8-baf6-82375000bfb3 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:28.899592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3f59f-8e36-42d8-baf6-82375000bfb3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:28.899592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3f59f-8e36-42d8-baf6-82375000bfb3 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:28.900826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 575/1170] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:28.906369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcaa1e-fbc6-42a7-86a3-96d6b465934b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:28.906879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcaa1e-fbc6-42a7-86a3-96d6b465934b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:28.907002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcaa1e-fbc6-42a7-86a3-96d6b465934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:28.924556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0fdcaa1e-fbc6-42a7-86a3-96d6b465934b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:28.925810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0fdcaa1e-fbc6-42a7-86a3-96d6b465934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:28.991889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 596/1171] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:29.015939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3f59f-8e36-42d8-baf6-82375000bfb3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:29.119066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3f59f-8e36-42d8-baf6-82375000bfb3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:29.154614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961b8c2-7d15-4eb6-a411-e0b7cceccee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:29.155387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961b8c2-7d15-4eb6-a411-e0b7cceccee8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:29.155672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961b8c2-7d15-4eb6-a411-e0b7cceccee8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:29.155941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961b8c2-7d15-4eb6-a411-e0b7cceccee8 None admin] PATH_INFO: `/v3/domains/8eb4285355834ed2be80022a4747d13e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:29.167113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961b8c2-7d15-4eb6-a411-e0b7cceccee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8eb4285355834ed2be80022a4747d13e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:29.168620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961b8c2-7d15-4eb6-a411-e0b7cceccee8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:29.170412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 576/1172] 10.222.0.32 () {56 vars in 1183 bytes} [Wed May 4 14:40:29 2022] GET /identity/v3/domains/8eb4285355834ed2be80022a4747d13e => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:40:29.176633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcaa1e-fbc6-42a7-86a3-96d6b465934b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:29.277061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcaa1e-fbc6-42a7-86a3-96d6b465934b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:29.311985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05eca2b-8f5d-4022-a311-6e25bb5780da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:29.312688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05eca2b-8f5d-4022-a311-6e25bb5780da None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:29.312982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05eca2b-8f5d-4022-a311-6e25bb5780da None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:29.313248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05eca2b-8f5d-4022-a311-6e25bb5780da None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:29.313961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05eca2b-8f5d-4022-a311-6e25bb5780da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:29.315388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05eca2b-8f5d-4022-a311-6e25bb5780da None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:29.344467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 597/1173] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:40:29 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:40:31.696333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9155d-6d55-4283-ac05-2e4623350d45 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:31.696961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9155d-6d55-4283-ac05-2e4623350d45 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:31.696961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9155d-6d55-4283-ac05-2e4623350d45 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:31.698260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 577/1174] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:31.704469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934959-8129-4ac6-9647-2f8cea1b083e None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:31.705047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934959-8129-4ac6-9647-2f8cea1b083e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:31.705047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934959-8129-4ac6-9647-2f8cea1b083e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:31.723636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-bc934959-8129-4ac6-9647-2f8cea1b083e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:31.724952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bc934959-8129-4ac6-9647-2f8cea1b083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:31.797441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 598/1175] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:31.822780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9155d-6d55-4283-ac05-2e4623350d45 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:31.929629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9155d-6d55-4283-ac05-2e4623350d45 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:31.966596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515384e-4de5-4f13-841d-bd8d2fd66c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:31.967411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515384e-4de5-4f13-841d-bd8d2fd66c3d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:31.967691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515384e-4de5-4f13-841d-bd8d2fd66c3d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:31.967964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515384e-4de5-4f13-841d-bd8d2fd66c3d None admin] PATH_INFO: `/v3/domains/8eb4285355834ed2be80022a4747d13e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:31.969317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515384e-4de5-4f13-841d-bd8d2fd66c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=8eb4285355834ed2be80022a4747d13e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:31.970761 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515384e-4de5-4f13-841d-bd8d2fd66c3d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:31.972438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 578/1176] 10.222.0.32 () {56 vars in 1183 bytes} [Wed May 4 14:40:31 2022] GET /identity/v3/domains/8eb4285355834ed2be80022a4747d13e => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:31.978616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc934959-8129-4ac6-9647-2f8cea1b083e None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:32.083992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc934959-8129-4ac6-9647-2f8cea1b083e None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:32.121241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b1b7c-e627-43c2-aa32-439f94060a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:32.121990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b1b7c-e627-43c2-aa32-439f94060a10 None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:32.122280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b1b7c-e627-43c2-aa32-439f94060a10 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:32.122593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b1b7c-e627-43c2-aa32-439f94060a10 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:32.123358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b1b7c-e627-43c2-aa32-439f94060a10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:32.124970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b1b7c-e627-43c2-aa32-439f94060a10 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:32.148988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 599/1177] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:40:31 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:40:34.677994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d57aad-8a01-43e9-aa76-876c3e9da7de None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:34.678569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d57aad-8a01-43e9-aa76-876c3e9da7de None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:34.678666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d57aad-8a01-43e9-aa76-876c3e9da7de None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:34.680189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 579/1178] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:34.687554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86195b6-31d0-4368-b6d2-90ff1c929005 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:34.688108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86195b6-31d0-4368-b6d2-90ff1c929005 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:34.688215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86195b6-31d0-4368-b6d2-90ff1c929005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:34.707239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d86195b6-31d0-4368-b6d2-90ff1c929005 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:34.708691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d86195b6-31d0-4368-b6d2-90ff1c929005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:34.783922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 600/1179] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:34.810418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d57aad-8a01-43e9-aa76-876c3e9da7de None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:34.930001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d57aad-8a01-43e9-aa76-876c3e9da7de None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:34.971466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:34.972293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:34.972604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:34.972949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:34.986802 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:40:34.995536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:40:34.997073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:40:34.998259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 580/1180] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:40:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:40:35.003384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86195b6-31d0-4368-b6d2-90ff1c929005 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:35.120065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86195b6-31d0-4368-b6d2-90ff1c929005 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:35.158302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:35.158985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:35.159264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:35.159542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:35.160309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:35.160611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:35.162077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:35.171633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 601/1181] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:40:35.181578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:35.321320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d9bea-6393-4e47-9ccf-1bcfe9ada9d1 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:35.361306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:35.362226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:35.362577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:35.362900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] PATH_INFO: `/v3/users/a744c89b1e054cafad98ff8ddf3a5b84` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:35.378579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] RBAC: Authorizing `identity:get_user(user_id=a744c89b1e054cafad98ff8ddf3a5b84)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:35.380147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:35.382026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 581/1182] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:35 2022] GET /identity/v3/users/a744c89b1e054cafad98ff8ddf3a5b84 => generated 311 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:35.388840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:35.493873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1bf06-b3ea-4d66-b8cb-281966810067 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:35.530885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:35.531637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:35.532078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:35.532403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:35.603474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:35.604970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:35.713533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 602/1183] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:40:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:40:35.720372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:35.873411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a090789-e493-4e80-952c-0f0d66a9f62f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:35.924721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd978f-160a-4a86-9c26-942808acb9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:35.926156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd978f-160a-4a86-9c26-942808acb9a0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:35.926701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd978f-160a-4a86-9c26-942808acb9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:35.927176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd978f-160a-4a86-9c26-942808acb9a0 None admin] PATH_INFO: `/v3/projects/ce11535a82a24feaad6efe4ce2781bb9` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:35.928833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd978f-160a-4a86-9c26-942808acb9a0 None admin] RBAC: Authorizing `identity:get_project(project_id=ce11535a82a24feaad6efe4ce2781bb9)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:35.930699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd978f-160a-4a86-9c26-942808acb9a0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:35.933171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 582/1184] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:35 2022] GET /identity/v3/projects/ce11535a82a24feaad6efe4ce2781bb9 => generated 361 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:35.943090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:36.074077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d5e00-c360-4c34-acb4-b84364640079 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:36.107695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472d07c-1bb1-42e8-9743-e012c1ecea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:36.108360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d07c-1bb1-42e8-9743-e012c1ecea65 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:36.108639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d07c-1bb1-42e8-9743-e012c1ecea65 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:36.108942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d07c-1bb1-42e8-9743-e012c1ecea65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:36.109788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472d07c-1bb1-42e8-9743-e012c1ecea65 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=ce11535a82a24feaad6efe4ce2781bb9, user.id=a744c89b1e054cafad98ff8ddf3a5b84) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:36.110096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472d07c-1bb1-42e8-9743-e012c1ecea65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:36.111553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472d07c-1bb1-42e8-9743-e012c1ecea65 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:36.117753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 603/1185] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:40:35 2022] GET /identity/v3/role_assignments?user.id=a744c89b1e054cafad98ff8ddf3a5b84&scope.project.id=ce11535a82a24feaad6efe4ce2781bb9&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:38.616815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f16b9-64f7-48de-a954-1247959694aa None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:38.617407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f16b9-64f7-48de-a954-1247959694aa None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:38.617407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f16b9-64f7-48de-a954-1247959694aa None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:38.618908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 583/1186] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:38.629218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcf76a-f669-4a11-92cc-6f405f7de319 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:38.629928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcf76a-f669-4a11-92cc-6f405f7de319 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:38.629928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcf76a-f669-4a11-92cc-6f405f7de319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:38.648681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-55fcf76a-f669-4a11-92cc-6f405f7de319 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:38.649977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-55fcf76a-f669-4a11-92cc-6f405f7de319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:38.721025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 604/1187] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:38.743892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f16b9-64f7-48de-a954-1247959694aa None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:38.852134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f16b9-64f7-48de-a954-1247959694aa None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:38.898710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:38.899455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:38.899729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:38.900000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:38.913210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:40:38.921152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:40:38.923229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:40:38.924826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 584/1188] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:40:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:40:38.929593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcf76a-f669-4a11-92cc-6f405f7de319 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:39.048120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcf76a-f669-4a11-92cc-6f405f7de319 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:39.087404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:39.088080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:39.088358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:39.088625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:39.089414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:39.089717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:39.091094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:39.100893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 605/1189] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:40:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:39.107354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:39.221104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943de4-51c3-4492-9a65-e16e1c3af265 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:39.258692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:39.259577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:39.259860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:39.260111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] PATH_INFO: `/v3/users/a744c89b1e054cafad98ff8ddf3a5b84` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:39.261546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=a744c89b1e054cafad98ff8ddf3a5b84)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:39.263075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:39.264882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 585/1190] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:39 2022] GET /identity/v3/users/a744c89b1e054cafad98ff8ddf3a5b84 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:39.271919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:39.383017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f54f84-bedf-411b-8e20-5610f61938f2 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:39.422212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a32621-a526-4812-a3d6-3ee9b1ce4dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:39.423354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a32621-a526-4812-a3d6-3ee9b1ce4dca None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:39.423784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a32621-a526-4812-a3d6-3ee9b1ce4dca None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:39.424309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a32621-a526-4812-a3d6-3ee9b1ce4dca None admin] PATH_INFO: `/v3/projects/ce11535a82a24feaad6efe4ce2781bb9` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:39.426429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a32621-a526-4812-a3d6-3ee9b1ce4dca None admin] RBAC: Authorizing `identity:get_project(project_id=ce11535a82a24feaad6efe4ce2781bb9)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:39.429419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a32621-a526-4812-a3d6-3ee9b1ce4dca None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:39.432519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 606/1191] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:39 2022] GET /identity/v3/projects/ce11535a82a24feaad6efe4ce2781bb9 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:39.444084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:39.576176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031448e-75cd-4830-8252-68199a3a2ed3 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:39.618493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73443f17-8ec5-41e4-be82-4a5730fb5ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:39.619415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443f17-8ec5-41e4-be82-4a5730fb5ab3 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:39.619703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443f17-8ec5-41e4-be82-4a5730fb5ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:39.619964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73443f17-8ec5-41e4-be82-4a5730fb5ab3 None admin] PATH_INFO: `/v3/projects/ce11535a82a24feaad6efe4ce2781bb9/users/a744c89b1e054cafad98ff8ddf3a5b84/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:39.622110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443f17-8ec5-41e4-be82-4a5730fb5ab3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce11535a82a24feaad6efe4ce2781bb9, user_id=a744c89b1e054cafad98ff8ddf3a5b84, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:39.623587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73443f17-8ec5-41e4-be82-4a5730fb5ab3 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:39.638447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 586/1192] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:40:39 2022] PUT /identity/v3/projects/ce11535a82a24feaad6efe4ce2781bb9/users/a744c89b1e054cafad98ff8ddf3a5b84/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:40:41.942219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfadccd-3211-4b9e-9cb1-b6fd212fd726 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:41.942801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfadccd-3211-4b9e-9cb1-b6fd212fd726 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:41.942801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfadccd-3211-4b9e-9cb1-b6fd212fd726 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:41.944147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 607/1193] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:40:41.950137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a4b2-16cb-4c90-baec-85484785c9c1 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:41.950680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a4b2-16cb-4c90-baec-85484785c9c1 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:41.950680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a4b2-16cb-4c90-baec-85484785c9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:41.968697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0e13a4b2-16cb-4c90-baec-85484785c9c1 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:40:41.969925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0e13a4b2-16cb-4c90-baec-85484785c9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:40:42.041136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 587/1194] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:40:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:40:42.065244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfadccd-3211-4b9e-9cb1-b6fd212fd726 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:42.167136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfadccd-3211-4b9e-9cb1-b6fd212fd726 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:42.200981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:42.201687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:42.201974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:42.202245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:42.213985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:40:42.221443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:40:42.222579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:40:42.223601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 608/1195] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:40:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:40:42.229058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a4b2-16cb-4c90-baec-85484785c9c1 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:42.331895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a4b2-16cb-4c90-baec-85484785c9c1 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:42.364600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:42.365300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:42.365576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:42.365850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:42.366593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:42.366890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:42.368243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:42.375989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 588/1196] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:40:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:42.382263 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:42.485502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03eb45-997b-4926-8390-4dca5713dfda None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:42.522310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:42.523209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:42.523489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:42.523760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] PATH_INFO: `/v3/users/a744c89b1e054cafad98ff8ddf3a5b84` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:42.525046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] RBAC: Authorizing `identity:get_user(user_id=a744c89b1e054cafad98ff8ddf3a5b84)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:42.526511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:42.528167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 609/1197] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 4 14:40:42 2022] GET /identity/v3/users/a744c89b1e054cafad98ff8ddf3a5b84 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:42.534352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:42.638784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f908d4e-cd60-4938-bffa-8a59474f5fbd None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:42.672908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:42.673515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:42.673795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:42.674064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:42.740274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:42.741671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:42.849161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 589/1198] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:40:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:40:42.856512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:42.965105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03504654-71d2-4a0f-9d3d-b48191e27f7a None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:42.999542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49e702-a2c7-48d1-a75e-bbb7c77e1bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:43.000319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49e702-a2c7-48d1-a75e-bbb7c77e1bed None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:43.000597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49e702-a2c7-48d1-a75e-bbb7c77e1bed None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:43.000895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49e702-a2c7-48d1-a75e-bbb7c77e1bed None admin] PATH_INFO: `/v3/projects/ce11535a82a24feaad6efe4ce2781bb9` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:43.002110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49e702-a2c7-48d1-a75e-bbb7c77e1bed None admin] RBAC: Authorizing `identity:get_project(project_id=ce11535a82a24feaad6efe4ce2781bb9)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:43.003540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49e702-a2c7-48d1-a75e-bbb7c77e1bed None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:43.005400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 610/1199] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 4 14:40:42 2022] GET /identity/v3/projects/ce11535a82a24feaad6efe4ce2781bb9 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:40:43.011898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:40:43.115845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bfd52-d933-44eb-a762-bf54b4701b0c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:40:43.152924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09da1f9-0c8f-41e0-b2c1-817b9cb8760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:40:43.153570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da1f9-0c8f-41e0-b2c1-817b9cb8760c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:40:43.153880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da1f9-0c8f-41e0-b2c1-817b9cb8760c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:40:43.154158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09da1f9-0c8f-41e0-b2c1-817b9cb8760c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:40:43.155005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09da1f9-0c8f-41e0-b2c1-817b9cb8760c None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=ce11535a82a24feaad6efe4ce2781bb9, user.id=a744c89b1e054cafad98ff8ddf3a5b84) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:40:43.155316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09da1f9-0c8f-41e0-b2c1-817b9cb8760c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:40:43.156832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09da1f9-0c8f-41e0-b2c1-817b9cb8760c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:40:43.163248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 590/1200] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:40:43 2022] GET /identity/v3/role_assignments?user.id=a744c89b1e054cafad98ff8ddf3a5b84&scope.project.id=ce11535a82a24feaad6efe4ce2781bb9&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:45.992703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cebb03-5a93-4ada-9371-3194de192d8c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:45.992703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cebb03-5a93-4ada-9371-3194de192d8c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:45.992703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cebb03-5a93-4ada-9371-3194de192d8c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:45.993685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 611/1201] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:41:46.001353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6ae48-e5c4-4049-8390-ec05180c3e2d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:46.002127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6ae48-e5c4-4049-8390-ec05180c3e2d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:46.002127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6ae48-e5c4-4049-8390-ec05180c3e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:46.026707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7cd6ae48-e5c4-4049-8390-ec05180c3e2d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:41:46.028634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7cd6ae48-e5c4-4049-8390-ec05180c3e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:41:46.128734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 591/1202] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:41:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:41:46.153537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cebb03-5a93-4ada-9371-3194de192d8c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:46.278745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cebb03-5a93-4ada-9371-3194de192d8c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:46.328343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:46.328343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:46.328343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:46.328343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:46.345227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:46.347069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:41:46.369267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:46.370396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:46.371462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:41:46.371462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:41:46.371462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:46.371462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:41:46.371462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 612/1203] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:46.378743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6ae48-e5c4-4049-8390-ec05180c3e2d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:46.512526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6ae48-e5c4-4049-8390-ec05180c3e2d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:46.562270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03db01-8718-422e-84eb-cb5c64a4fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:46.563310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03db01-8718-422e-84eb-cb5c64a4fa21 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:46.563843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03db01-8718-422e-84eb-cb5c64a4fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:46.564309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03db01-8718-422e-84eb-cb5c64a4fa21 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:46.565707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03db01-8718-422e-84eb-cb5c64a4fa21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:46.566257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03db01-8718-422e-84eb-cb5c64a4fa21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:46.568842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03db01-8718-422e-84eb-cb5c64a4fa21 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:46.595047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 592/1204] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:41:46.605252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:46.793516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d56714-4df2-46a3-bec9-0b83f9f0299d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:46.859337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ef828-52eb-4c37-9f9f-a7b1ffb1a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:46.860476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ef828-52eb-4c37-9f9f-a7b1ffb1a14f None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:46.860970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ef828-52eb-4c37-9f9f-a7b1ffb1a14f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:46.861447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ef828-52eb-4c37-9f9f-a7b1ffb1a14f None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:46.862807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ef828-52eb-4c37-9f9f-a7b1ffb1a14f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:46.865522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ef828-52eb-4c37-9f9f-a7b1ffb1a14f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:46.897832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 613/1205] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 4 14:41:46 2022] POST /identity/v3/users => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:41:49.671210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119dddf-3844-48b4-8682-0fa318c61ed9 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:49.671956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119dddf-3844-48b4-8682-0fa318c61ed9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:49.671956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119dddf-3844-48b4-8682-0fa318c61ed9 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:49.673212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 593/1206] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:41:49.679764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222776e7-8fd1-487d-adc4-e654dfbef625 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:49.680362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222776e7-8fd1-487d-adc4-e654dfbef625 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:49.680362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222776e7-8fd1-487d-adc4-e654dfbef625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:49.699922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-222776e7-8fd1-487d-adc4-e654dfbef625 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:41:49.701184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-222776e7-8fd1-487d-adc4-e654dfbef625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:41:49.780753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 614/1207] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:41:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:41:49.804596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119dddf-3844-48b4-8682-0fa318c61ed9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:49.950617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119dddf-3844-48b4-8682-0fa318c61ed9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:49.999222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:50.000257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:50.000594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:50.000925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:50.014072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:41:50.023755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:50.026396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:50.027432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 594/1208] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:41:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:50.032310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222776e7-8fd1-487d-adc4-e654dfbef625 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:50.154434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222776e7-8fd1-487d-adc4-e654dfbef625 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:50.206756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:50.207394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:50.207675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:50.208039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:50.209845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:50.209845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:50.210793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:50.231416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 615/1209] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:41:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:50.233356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:50.366251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8feecab-db70-4052-a85e-50ab636a08b2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:50.412581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:50.413444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:50.413727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:50.414009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:50.427214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:50.428639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:41:50.451436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:50.452548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:50.453528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:41:50.453528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:41:50.453528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:50.453528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:50.453528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 595/1210] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:41:50.459595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:50.579677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b823fdb-5b94-488e-8178-6c8cd5ccfdf1 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:50.622797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:50.624211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:50.624907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:50.625574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:50.627125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:50.627844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:50.630514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:50.656168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 616/1211] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:50.658780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:50.811916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cf2a4-ad16-420e-8342-8dd59b3ad29a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:50.858692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:50.859632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:50.859943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:50.860288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:50.870149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:50.871683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:41:50.888856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:50.889997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:50.890970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 596/1212] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:41:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:41:50.896975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:51.028334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef379aa-6bc1-4e26-bac2-d39b7c977712 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:51.069724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:51.070403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:51.070680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:51.070941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:51.071875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:51.072190 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:51.073748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:51.094159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 617/1213] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 4 14:41:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:41:51.100343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:51.220307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84156715-bfb7-4a34-bd73-650ad46cc80c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:51.259381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:51.260341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:51.262063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:51.262063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:51.348921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:51.351685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:51.516006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 597/1214] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:41:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:41:51.525898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:51.683135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23b2f-5079-4b01-961e-1fc0fd9422aa None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:51.728730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:51.729653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:51.729928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:51.730271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:51.742569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:51.744292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:41:51.765634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:51.769636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:51.771880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:41:51.771880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:41:51.771880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:51.771880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:41:51.771880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 618/1215] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:51.773745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:51.878977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e75cfe-2536-4c1b-9e42-48ebf1517f1a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:51.919693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:51.920374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:51.920655 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:51.920959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:51.921718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:51.922015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:51.923523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:51.935830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 598/1216] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:41:51.941592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:52.056989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970845a-fff7-4bd3-80d3-93c02ee9eb92 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:52.098632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:52.098632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:52.098981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:52.098981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:52.106623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:52.108230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:41:52.124639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:52.125926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:41:52.126884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 619/1217] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:52.133283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:52.248242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669563-c589-4c76-a046-46b3a6345cb2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:52.287289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33233c71-949b-46ac-8031-4619c6a1abc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:52.287995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233c71-949b-46ac-8031-4619c6a1abc1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:52.288308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233c71-949b-46ac-8031-4619c6a1abc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:52.288550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33233c71-949b-46ac-8031-4619c6a1abc1 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:52.289413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233c71-949b-46ac-8031-4619c6a1abc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:52.289729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233c71-949b-46ac-8031-4619c6a1abc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:52.291345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33233c71-949b-46ac-8031-4619c6a1abc1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:52.305152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 599/1218] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:41:52.314630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:52.431520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ae564-8d13-4da0-8a80-89bb5f320ad6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:52.470538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164616b-0ef0-44b4-8cba-fede17b07ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:52.471257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164616b-0ef0-44b4-8cba-fede17b07ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:52.471509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164616b-0ef0-44b4-8cba-fede17b07ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:52.471768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164616b-0ef0-44b4-8cba-fede17b07ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:52.472654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164616b-0ef0-44b4-8cba-fede17b07ec7 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=7ac21bca8da642338571f821ef5ea121) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:52.472971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164616b-0ef0-44b4-8cba-fede17b07ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:52.474446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164616b-0ef0-44b4-8cba-fede17b07ec7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:52.489078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 620/1219] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:41:52 2022] GET /identity/v3/role_assignments?user.id=7ac21bca8da642338571f821ef5ea121&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:54.912998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e634a9-8917-481e-bd8f-fdbf4ed9f70e None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:54.913599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e634a9-8917-481e-bd8f-fdbf4ed9f70e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:54.913599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e634a9-8917-481e-bd8f-fdbf4ed9f70e None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:54.914968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 600/1220] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:41:54.920689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129f60c-eb9d-4310-81ca-4fd69907a8a3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:54.921319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129f60c-eb9d-4310-81ca-4fd69907a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:54.921319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129f60c-eb9d-4310-81ca-4fd69907a8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:54.939260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a129f60c-eb9d-4310-81ca-4fd69907a8a3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:41:54.940466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a129f60c-eb9d-4310-81ca-4fd69907a8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:41:55.016179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 621/1221] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:41:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:41:55.042955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e634a9-8917-481e-bd8f-fdbf4ed9f70e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:55.157157 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e634a9-8917-481e-bd8f-fdbf4ed9f70e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:55.200586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:55.201640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:55.201979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:55.202431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:55.215772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:41:55.224147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:55.225347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:55.231677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 601/1222] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:41:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:55.232472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129f60c-eb9d-4310-81ca-4fd69907a8a3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:55.362232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129f60c-eb9d-4310-81ca-4fd69907a8a3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:55.404380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:55.405178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:55.405541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:55.405885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:55.407039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] RBAC: Adding query filter params (name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:55.407370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:55.409404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:55.419841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 622/1223] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:41:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:55.428005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:55.557707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a87eaf-122c-4d23-85de-33dee26f8e18 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:55.597718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:55.598533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:55.598825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:55.599086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:55.611851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:55.613542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:41:55.632823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:55.634126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:55.635620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:41:55.635620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:41:55.635620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:55.635620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:55.635620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 602/1224] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:55.640837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:55.754981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18936a-4266-442a-ac84-b7caca9ed910 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:55.790313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:55.791102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:55.791369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:55.791667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:55.792432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:55.792766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:55.794377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:55.807264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 623/1225] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:55.813336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:55.925119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7003e-9e79-4949-a3cf-c01dc73b2e8b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:55.962894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:55.963821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:55.964135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:55.964434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] PATH_INFO: `/v3/users/placement` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:55.974558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:55.976262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:41:55.992227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:55.993529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:55.994833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 603/1226] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:41:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:55.999882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:56.113046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91167947-95ea-412c-ac36-6b47255a918f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:56.154998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:56.155682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:56.155965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:56.156239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:56.157180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:56.157476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:56.158932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:56.180063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 624/1227] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 4 14:41:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:56.187680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:56.319984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52938e1-e082-4ce8-b15a-f79e94099b0d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:56.359667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:56.360595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:56.360904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:56.361182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:56.374498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:56.376279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:41:56.398574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:56.401570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:56.402522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:41:56.402522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:41:56.402522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:41:56.402522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:56.402522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 604/1228] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:41:56.406464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:56.529680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e1ea9-fc37-44d5-a574-5fc22bd934e1 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:56.580351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:56.581075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:56.581838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:56.581838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:56.582373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:56.582684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:56.584365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:56.597806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 625/1229] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:41:56.603897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:56.730925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78b232-38c2-4a73-9f65-57aec5019abe None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:56.781094 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:56.781994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:56.782200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:56.782485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:56.789814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:56.791356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:41:56.806587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:56.807854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:41:56.808831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 605/1230] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:56.814072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:56.940165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06992e2d-e253-4ed2-b1a9-6e1aa360cc3f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:56.986361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d2736-2452-4e9b-b74c-86f2aab99cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:56.987138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2736-2452-4e9b-b74c-86f2aab99cef None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:56.987417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2736-2452-4e9b-b74c-86f2aab99cef None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:56.987825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2736-2452-4e9b-b74c-86f2aab99cef None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:56.988755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d2736-2452-4e9b-b74c-86f2aab99cef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:41:56.989224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d2736-2452-4e9b-b74c-86f2aab99cef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:56.991116 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d2736-2452-4e9b-b74c-86f2aab99cef None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:57.007750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 626/1231] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:41:57.016048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:57.134675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67179fbe-546c-4f67-8b98-1007b405a88f None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:57.180624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33c4df-a02a-41ed-968b-119b403094e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:57.181861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33c4df-a02a-41ed-968b-119b403094e9 None admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:57.182164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33c4df-a02a-41ed-968b-119b403094e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:57.182813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33c4df-a02a-41ed-968b-119b403094e9 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/7ac21bca8da642338571f821ef5ea121/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:57.200905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33c4df-a02a-41ed-968b-119b403094e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=7ac21bca8da642338571f821ef5ea121, role_id=ac8e88511fc54ccda9cfa0f2e06cdaa3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:41:57.202781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33c4df-a02a-41ed-968b-119b403094e9 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:41:57.216703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 606/1232] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:41:57 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/7ac21bca8da642338571f821ef5ea121/roles/ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:41:59.585091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b48ca-f52f-43c4-93b0-228fc23664e3 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:59.585091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b48ca-f52f-43c4-93b0-228fc23664e3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:59.585806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b48ca-f52f-43c4-93b0-228fc23664e3 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:59.586532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 627/1233] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:41:59.592778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03221-db2b-47cd-b02b-e5cd3ee792bd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:59.593307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03221-db2b-47cd-b02b-e5cd3ee792bd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:59.593420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03221-db2b-47cd-b02b-e5cd3ee792bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:59.610908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3fd03221-db2b-47cd-b02b-e5cd3ee792bd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:41:59.612177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3fd03221-db2b-47cd-b02b-e5cd3ee792bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:41:59.688862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 607/1234] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:41:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:41:59.716625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b48ca-f52f-43c4-93b0-228fc23664e3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:41:59.849017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b48ca-f52f-43c4-93b0-228fc23664e3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:41:59.898413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:41:59.899807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:41:59.900313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:41:59.900854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:41:59.913988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:41:59.921974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:41:59.923403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:41:59.924539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 628/1235] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 4 14:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:41:59.932501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd03221-db2b-47cd-b02b-e5cd3ee792bd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:00.048609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd03221-db2b-47cd-b02b-e5cd3ee792bd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:00.084576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:00.085282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:00.085562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:00.085836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:00.086594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:00.087036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:00.088886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:00.098829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 608/1236] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 4 14:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:00.109782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:00.226687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca97bde-e12c-4374-813d-8bf0f13a35d9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:00.271845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:00.272671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:00.272989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:00.273444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:00.287716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:00.289201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:00.310285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:00.311651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:00.313282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:00.313282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:00.313282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:00.313282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:00.313282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 629/1237] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:00.322005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:00.433727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25636cdd-5b56-4f33-a30d-1de8140a4cc0 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:00.471096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:00.471781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:00.472067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:00.472350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:00.473188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:00.473535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:00.474987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:00.490409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 609/1238] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:00.496667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:00.614022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118769ef-70b7-43a9-8058-2a4f7912610d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:00.658471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:00.659372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:00.659628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:00.659942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] PATH_INFO: `/v3/users/placement` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:00.670095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:00.671631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:42:00.689967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:00.691783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:00.693058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 630/1239] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:00.698088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:00.842339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6780b9-0356-4fb6-9434-62ea1ee8086c None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:00.884269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:00.885445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:00.885848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:00.886350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:00.887757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:00.888136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:00.891067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:00.913381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 610/1240] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 4 14:42:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:00.920078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:01.035984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb4a13-d4c4-4102-987b-a9415718ebbb None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:01.072196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:01.072916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:01.073205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:01.073494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:01.145754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:01.147423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:01.257842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 631/1241] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:42:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:42:01.259115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:01.378539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0712fdd-977d-4c6d-9013-d15d1b7ee8e2 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:01.422902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:01.423683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:01.423962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:01.424279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:01.435809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:01.437339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:01.455748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:01.457134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:01.458296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:01.458296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:01.458296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:01.458296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:01.458296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 611/1242] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 04 14:42:01.463242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:01.573448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00bd56-563f-4c5e-b6b0-78ae7732e1a9 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:01.612233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:01.612985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:01.613270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:01.613544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:01.614319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:01.614889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:01.616135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:01.628662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 632/1243] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:01.634647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:01.757942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db153adb-11d7-4c70-84f1-62adaef33013 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:01.793040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:01.793865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:01.794140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:01.794415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:01.801903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:01.803393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:42:01.822113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:01.823334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:01.824646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 612/1244] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:01.830031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:01.955571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ffe15-ce47-4fac-b46d-d04528b842a7 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:01.991900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dd7b8-0736-4f5e-988f-af1e952dedaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:01.992565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd7b8-0736-4f5e-988f-af1e952dedaa None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:01.992865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd7b8-0736-4f5e-988f-af1e952dedaa None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:01.993135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd7b8-0736-4f5e-988f-af1e952dedaa None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:01.993923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dd7b8-0736-4f5e-988f-af1e952dedaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:01.994257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dd7b8-0736-4f5e-988f-af1e952dedaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:01.995860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dd7b8-0736-4f5e-988f-af1e952dedaa None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:02.010729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 633/1245] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:02.019161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:02.135540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f47646-07fd-4e15-a98c-348845cf03f4 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:02.177756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec26b38-0008-4628-807c-3e4d8fb9d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:02.178482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26b38-0008-4628-807c-3e4d8fb9d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:02.178805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26b38-0008-4628-807c-3e4d8fb9d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:02.179338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec26b38-0008-4628-807c-3e4d8fb9d9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:02.180499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec26b38-0008-4628-807c-3e4d8fb9d9c1 None admin] RBAC: Adding query filter params (role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=7ac21bca8da642338571f821ef5ea121) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:02.180864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec26b38-0008-4628-807c-3e4d8fb9d9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:02.182482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec26b38-0008-4628-807c-3e4d8fb9d9c1 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:02.191096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 613/1246] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:42:02 2022] GET /identity/v3/role_assignments?user.id=7ac21bca8da642338571f821ef5ea121&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=ac8e88511fc54ccda9cfa0f2e06cdaa3 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:04.931084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfb353-8690-4bac-831b-36229cc8775c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:04.931915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfb353-8690-4bac-831b-36229cc8775c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:04.931915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfb353-8690-4bac-831b-36229cc8775c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:04.933644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 634/1247] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:42:04.939867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc27e1-6da2-4c3e-beac-726f0bf6d50b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:04.940657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc27e1-6da2-4c3e-beac-726f0bf6d50b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:04.940657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc27e1-6da2-4c3e-beac-726f0bf6d50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:04.961168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-92fc27e1-6da2-4c3e-beac-726f0bf6d50b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:42:04.962375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-92fc27e1-6da2-4c3e-beac-726f0bf6d50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:42:05.054584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 614/1248] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:42:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:42:05.080748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfb353-8690-4bac-831b-36229cc8775c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:05.199906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfb353-8690-4bac-831b-36229cc8775c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:05.238050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:05.238854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:05.239132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:05.239405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:05.251870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:42:05.260498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:05.261613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:05.262638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 635/1249] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:42:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:05.269145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc27e1-6da2-4c3e-beac-726f0bf6d50b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:05.392478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc27e1-6da2-4c3e-beac-726f0bf6d50b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:05.432630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:05.433413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:05.433747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:05.434095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:05.434929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:05.435286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:05.436768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:05.451906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 615/1250] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:42:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:05.454996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:05.570678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc135250-eb8b-42c8-bd7b-849dd5b41216 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:05.627259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:05.628374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:05.628600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:05.628973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:05.644544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:05.646835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:05.667382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:05.669184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:05.670879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:05.670879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:05.670879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:05.670879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:05.670879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 636/1251] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:05.678421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:05.822149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc5bb-6e06-40e8-a4fb-a7dbbf94ae63 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:05.866953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da7981-5324-43b0-8878-a2436572005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:05.867835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da7981-5324-43b0-8878-a2436572005b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:05.868215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da7981-5324-43b0-8878-a2436572005b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:05.868382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da7981-5324-43b0-8878-a2436572005b None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:05.869345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da7981-5324-43b0-8878-a2436572005b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:05.869680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da7981-5324-43b0-8878-a2436572005b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:05.871129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da7981-5324-43b0-8878-a2436572005b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:05.887456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 616/1252] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:05.895405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:06.048626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337edc5-f476-4e9f-a76a-29be2d9e6f3f None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:06.090461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:06.091323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:06.091637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:06.092012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:06.101008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:06.102479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:42:06.117654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:06.118939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:06.119950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 637/1253] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:06.125474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da7981-5324-43b0-8878-a2436572005b None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:06.242170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da7981-5324-43b0-8878-a2436572005b None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:06.283509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:06.284271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:06.284555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:06.284867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:06.285783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:06.286112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:06.287692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:06.308386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 617/1254] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 4 14:42:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:06.313618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:06.432684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbb7e7-4bbd-4436-a104-ba904bd8e9f6 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:06.470412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:06.471306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:06.471713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:06.472038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:06.558452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:06.560458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:06.698405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 638/1255] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:42:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 04 14:42:06.700109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:06.822275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4161907-074c-426c-9e5f-cbd98ce9d9a8 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:06.861294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:06.862113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:06.862410 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:06.862661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:06.873780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:06.875414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:06.894620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:06.895821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:06.896963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:06.896963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:06.896963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:06.896963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:06.896963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 618/1256] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:06.902257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:07.016270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e14da-4d16-4d72-a4a2-370e7ed70487 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:07.053652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:07.054369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:07.054596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:07.054870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:07.055608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:07.055893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:07.057359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:07.069828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 639/1257] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:07.075758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:07.186673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c745c-d0ad-4227-9cc0-cecf405866ab None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:07.221472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:07.222254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:07.222532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:07.222800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:07.230131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:07.231572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:42:07.245981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:07.247052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:07.247960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 619/1258] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:07.253917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:07.386749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c7fe5-7306-42af-87c4-eb2e98fff94b None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:07.424764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5543e27-f9b4-4e4c-af68-1a0daa9bad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:07.425486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543e27-f9b4-4e4c-af68-1a0daa9bad5a None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:07.425761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543e27-f9b4-4e4c-af68-1a0daa9bad5a None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:07.426029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543e27-f9b4-4e4c-af68-1a0daa9bad5a None admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:07.426837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5543e27-f9b4-4e4c-af68-1a0daa9bad5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:07.427139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5543e27-f9b4-4e4c-af68-1a0daa9bad5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:07.428582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5543e27-f9b4-4e4c-af68-1a0daa9bad5a None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:07.442477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 640/1259] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:42:07.448973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:07.548931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8c1f-06ed-468a-8267-20c9b260f967 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:07.584471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951ee0a-2f34-45a2-9de5-a972b272ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:07.585066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951ee0a-2f34-45a2-9de5-a972b272ef0b None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:07.585344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951ee0a-2f34-45a2-9de5-a972b272ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:07.585609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951ee0a-2f34-45a2-9de5-a972b272ef0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:07.586443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951ee0a-2f34-45a2-9de5-a972b272ef0b None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=7ac21bca8da642338571f821ef5ea121) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:07.586775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951ee0a-2f34-45a2-9de5-a972b272ef0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:07.588447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951ee0a-2f34-45a2-9de5-a972b272ef0b None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:07.595090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 620/1260] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:42:07 2022] GET /identity/v3/role_assignments?user.id=7ac21bca8da642338571f821ef5ea121&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:09.884779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86309f23-0138-4750-b1cd-8fd191a255c9 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:09.885362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86309f23-0138-4750-b1cd-8fd191a255c9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:09.885362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86309f23-0138-4750-b1cd-8fd191a255c9 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:09.886709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 641/1261] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:42:09.892245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a6cf-6bc3-4f8f-865f-36977ede7586 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:09.893076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a6cf-6bc3-4f8f-865f-36977ede7586 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:09.893076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951a6cf-6bc3-4f8f-865f-36977ede7586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:09.910236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-2951a6cf-6bc3-4f8f-865f-36977ede7586 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:42:09.911434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2951a6cf-6bc3-4f8f-865f-36977ede7586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:42:09.980608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 621/1262] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:42:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:42:10.003329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86309f23-0138-4750-b1cd-8fd191a255c9 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:10.116515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86309f23-0138-4750-b1cd-8fd191a255c9 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:10.155032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:10.155742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:10.156093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:10.156417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:10.168847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:42:10.176902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:10.177995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:10.178884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 642/1263] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:42:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:10.185457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a6cf-6bc3-4f8f-865f-36977ede7586 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:10.300969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951a6cf-6bc3-4f8f-865f-36977ede7586 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:10.346808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:10.347472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:10.347744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:10.348012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:10.348765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:10.349102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:10.350454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:10.359420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 622/1264] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:42:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:10.367565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:10.478071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a5bf7-85f7-4c22-8f19-d0530e9bd946 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:10.512763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:10.513684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:10.513967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:10.514333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:10.526504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:10.527977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:10.546618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:10.549933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:10.551069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:10.551069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:10.551069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:10.551069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:10.551069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 643/1265] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:10.554041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:10.668011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305d28f-fd0f-4257-82fe-fb3ebf19cb0d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:10.706612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:10.707261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:10.707545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:10.707816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:10.708546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:10.708876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:10.710256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:10.724657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 623/1266] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:10.731159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:10.858714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4885f-d970-4c0e-a7b0-88db577a1477 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:10.897576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:10.898374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:10.898651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:10.898923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:10.907291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:10.908747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:42:10.923450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:10.924609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:10.925598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 644/1267] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:10.930674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:11.030367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ee38-c151-418c-8ae1-dc54ba1ab59d None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:11.071358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:11.073304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:11.073304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:11.073304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:11.073691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:11.073809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:11.075201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:11.093497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 624/1268] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 4 14:42:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:42:11.099555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:11.205577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91810f-6320-459f-a913-65bd54ad4d8c None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:11.239575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:11.240434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:11.240756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:11.241132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:11.253010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:11.254519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:11.276303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:11.277941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:11.279674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:11.279674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:11.279674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:11.279674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:11.279674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 645/1269] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:11.284228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:11.384875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc4b4-5608-4c52-8ce5-e8132bc18294 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:11.419728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:11.420390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:11.420665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:11.420965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:11.421706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:11.421981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:11.423376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:11.435392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 625/1270] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:42:11.441753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:11.543729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118f369-0a77-438d-8a42-50dc6ba6a0df None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:11.580553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:11.581384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:11.581665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:11.581939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:11.589214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:11.590711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:42:11.605275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:11.606344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:11.607317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 646/1271] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:42:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:11.612611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:11.720135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925d75-8825-471e-a63b-adf6a07d756e None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:11.754686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadaff0b-2603-42eb-870f-bbcedd456570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:11.755360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaff0b-2603-42eb-870f-bbcedd456570 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:11.755645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaff0b-2603-42eb-870f-bbcedd456570 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:11.755915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaff0b-2603-42eb-870f-bbcedd456570 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:11.756714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadaff0b-2603-42eb-870f-bbcedd456570 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:11.757050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadaff0b-2603-42eb-870f-bbcedd456570 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:11.758482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadaff0b-2603-42eb-870f-bbcedd456570 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:11.771594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 626/1272] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:42:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:11.778075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:11.886969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f1952-7a62-4f5f-91bf-e888e2efa3af None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:11.924836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f91838-e2d1-43b4-ac61-88ff8758eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:11.925762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f91838-e2d1-43b4-ac61-88ff8758eaf1 None admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:11.926038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f91838-e2d1-43b4-ac61-88ff8758eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:11.926313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f91838-e2d1-43b4-ac61-88ff8758eaf1 None admin] PATH_INFO: `/v3/projects/79beb601016d422faea4e6a5753cad40/users/7ac21bca8da642338571f821ef5ea121/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:11.928393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f91838-e2d1-43b4-ac61-88ff8758eaf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=79beb601016d422faea4e6a5753cad40, user_id=7ac21bca8da642338571f821ef5ea121, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:11.929901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f91838-e2d1-43b4-ac61-88ff8758eaf1 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:11.945003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 647/1273] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 4 14:42:11 2022] PUT /identity/v3/projects/79beb601016d422faea4e6a5753cad40/users/7ac21bca8da642338571f821ef5ea121/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:42:14.180103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbb09e-2018-4d37-a698-a2c7d5497c20 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:14.180702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbb09e-2018-4d37-a698-a2c7d5497c20 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:14.180702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbb09e-2018-4d37-a698-a2c7d5497c20 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:14.182044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 627/1274] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:42:14.187393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d94fc-3dc4-4bac-9ef0-a8232f0b54d0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:14.187835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d94fc-3dc4-4bac-9ef0-a8232f0b54d0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:14.187933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d94fc-3dc4-4bac-9ef0-a8232f0b54d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:14.205314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0c1d94fc-3dc4-4bac-9ef0-a8232f0b54d0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:42:14.206509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0c1d94fc-3dc4-4bac-9ef0-a8232f0b54d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:42:14.276314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 648/1275] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:42:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:42:14.301837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbb09e-2018-4d37-a698-a2c7d5497c20 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:14.415101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbb09e-2018-4d37-a698-a2c7d5497c20 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:14.453038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:14.453739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:14.454020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:14.454295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:14.465139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(role_id) May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 04 14:42:14.472738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:14.473822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:14.474799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 628/1276] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 4 14:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:14.480345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d94fc-3dc4-4bac-9ef0-a8232f0b54d0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:14.575785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d94fc-3dc4-4bac-9ef0-a8232f0b54d0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:14.611718 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:14.612384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:14.612657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:14.612962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:14.613718 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:14.614021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:14.615401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:14.624724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 649/1277] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 4 14:42:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:14.630753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:14.747311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12346750-e2cf-444c-9bfa-9244a74c33bf None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:14.787138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:14.787996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:14.788284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:14.788561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:14.801646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:14.803254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:14.823283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:14.824409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:14.830453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:14.830453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:14.830453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:14.830453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:14.830453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 629/1278] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:14.831244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:14.942239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02ceb3-af54-4367-9de3-42343d7edaaf None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:14.977932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:14.978616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:14.978895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:14.979169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:14.979925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:14.980232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:14.981694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:14.995133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 650/1279] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:15.003321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:15.108097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171a475-a6bf-4128-8d39-4d99c13c7189 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:15.146967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:15.147982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:15.148376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:15.148737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:15.157960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:15.159496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_user(user_id) May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 04 14:42:15.174891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:15.176198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:15.178053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 630/1280] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:15.182302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:15.290409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ecb10-c71d-4f4a-a072-b8ebe301ea3d None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:15.330562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:15.331234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:15.331511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:15.331851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:15.332702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:15.333062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:15.334496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:15.352407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 651/1281] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 4 14:42:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:42:15.360727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:15.469227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81868-faa3-4b97-958f-6c393d1bff07 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:15.504774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:15.505458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:15.505742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:15.506018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:15.575349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:15.576721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:15.677718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 631/1282] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 4 14:42:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 04 14:42:15.684043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:15.807045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b81a2-6792-4754-b1b4-f14e4ded194e None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:15.844074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:15.844782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:15.845098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:15.845371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:15.855664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:15.857182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 04 14:42:15.882467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:15.884003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:15.885067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 04 14:42:15.885067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 04 14:42:15.885067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 04 14:42:15.885067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:15.885067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 652/1283] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:15.889564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:15.996569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389d9e9-d2bb-4258-a86e-43a3ce351a12 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:16.031942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:16.032612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:16.032921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:16.033198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:16.033931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:16.034232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:16.035651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:16.050585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 632/1284] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 4 14:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:16.056547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:16.175707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448ff5-9d6b-48d9-9301-d1b9f011a926 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:16.215060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:16.215872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:16.216154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:16.216423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:16.224603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:16.226120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(project_id) May 04 14:42:16.240981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application with Lock( May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:16.242136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: ERROR keystone.server.flask.application May 04 14:42:16.243389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 653/1285] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 4 14:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:16.248483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:16.351752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6d305-ad84-4712-b260-971c558ab535 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:16.387466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591d101-abbb-4ba8-ae68-a11a96340c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:16.388132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591d101-abbb-4ba8-ae68-a11a96340c48 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:16.388396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591d101-abbb-4ba8-ae68-a11a96340c48 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:16.388666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591d101-abbb-4ba8-ae68-a11a96340c48 None admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:16.389481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591d101-abbb-4ba8-ae68-a11a96340c48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:16.389792 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591d101-abbb-4ba8-ae68-a11a96340c48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:16.391269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591d101-abbb-4ba8-ae68-a11a96340c48 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:16.404045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 633/1286] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 4 14:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:16.411130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:16.521286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca99a3-3be2-44f7-a08d-d6b0cdd72614 None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:16.555444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7e19c-812b-4f4b-80f8-21c9d46580b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:16.556068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e19c-812b-4f4b-80f8-21c9d46580b2 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:16.556369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e19c-812b-4f4b-80f8-21c9d46580b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:16.556634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7e19c-812b-4f4b-80f8-21c9d46580b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:16.557507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7e19c-812b-4f4b-80f8-21c9d46580b2 None admin] RBAC: Adding query filter params (role.id=11ee08052a1446a2adaf40beef1e04c3, scope.project.id=79beb601016d422faea4e6a5753cad40, user.id=7ac21bca8da642338571f821ef5ea121) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:16.557814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7e19c-812b-4f4b-80f8-21c9d46580b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:16.559243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7e19c-812b-4f4b-80f8-21c9d46580b2 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:16.565705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 654/1287] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 4 14:42:16 2022] GET /identity/v3/role_assignments?user.id=7ac21bca8da642338571f821ef5ea121&scope.project.id=79beb601016d422faea4e6a5753cad40&role.id=11ee08052a1446a2adaf40beef1e04c3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:18.859762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a8f57-586f-413d-b234-08dc3bc1094b None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:18.860340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a8f57-586f-413d-b234-08dc3bc1094b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:18.860340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a8f57-586f-413d-b234-08dc3bc1094b None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:18.861731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 634/1288] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:42:18.877232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82693156-909b-473e-b730-f8b00fc00708 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:18.878099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82693156-909b-473e-b730-f8b00fc00708 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:18.878099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82693156-909b-473e-b730-f8b00fc00708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:18.897107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-82693156-909b-473e-b730-f8b00fc00708 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:42:18.898500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-82693156-909b-473e-b730-f8b00fc00708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:42:18.966716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 655/1289] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:42:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:42:18.989470 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a8f57-586f-413d-b234-08dc3bc1094b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:19.094565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a8f57-586f-413d-b234-08dc3bc1094b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:19.132549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:19.133303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:19.133582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:19.133848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] PATH_INFO: `/v3/services/placement` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:19.134502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:19.136003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:19.148984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:19.150338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:19.151231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 635/1290] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:42:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:19.157253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82693156-909b-473e-b730-f8b00fc00708 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:19.280763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82693156-909b-473e-b730-f8b00fc00708 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:19.315480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779239c3-9ed4-41f1-9849-c592aaf7ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:19.316231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779239c3-9ed4-41f1-9849-c592aaf7ca1b None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:19.316575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779239c3-9ed4-41f1-9849-c592aaf7ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:19.316991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779239c3-9ed4-41f1-9849-c592aaf7ca1b None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:19.317854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779239c3-9ed4-41f1-9849-c592aaf7ca1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:19.318210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779239c3-9ed4-41f1-9849-c592aaf7ca1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:19.319671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779239c3-9ed4-41f1-9849-c592aaf7ca1b None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:19.325615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 656/1291] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 4 14:42:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:19.331867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:19.434342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e6ade-732b-4923-9165-11604ddcd8df None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:19.471261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6030c26-624c-45f3-a944-99d324fe2209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:19.471947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6030c26-624c-45f3-a944-99d324fe2209 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:19.472231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6030c26-624c-45f3-a944-99d324fe2209 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:19.472504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6030c26-624c-45f3-a944-99d324fe2209 None admin] PATH_INFO: `/v3/services` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:19.473249 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6030c26-624c-45f3-a944-99d324fe2209 None admin] RBAC: Adding query filter params (type=placement) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:19.473557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6030c26-624c-45f3-a944-99d324fe2209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:19.474944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6030c26-624c-45f3-a944-99d324fe2209 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:19.480137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 636/1292] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 4 14:42:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:42:21.843872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25503c-d033-4431-bbf6-a8bb7c24e582 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:21.844526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25503c-d033-4431-bbf6-a8bb7c24e582 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:21.844913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25503c-d033-4431-bbf6-a8bb7c24e582 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:21.847451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 657/1293] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:42:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:42:21.853439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2f70e-a796-46f4-ab14-d0b9ec58c36d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:21.853951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2f70e-a796-46f4-ab14-d0b9ec58c36d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:21.854111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2f70e-a796-46f4-ab14-d0b9ec58c36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:21.873211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-afc2f70e-a796-46f4-ab14-d0b9ec58c36d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:42:21.874561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-afc2f70e-a796-46f4-ab14-d0b9ec58c36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:42:21.962090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 637/1294] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:42:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:42:21.986277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25503c-d033-4431-bbf6-a8bb7c24e582 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:22.094975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25503c-d033-4431-bbf6-a8bb7c24e582 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:22.133965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a8f04-5710-40f7-b860-61310127dafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:22.134638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a8f04-5710-40f7-b860-61310127dafd None admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:22.134915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a8f04-5710-40f7-b860-61310127dafd None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:22.135185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a8f04-5710-40f7-b860-61310127dafd None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:22.135947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a8f04-5710-40f7-b860-61310127dafd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:22.137377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a8f04-5710-40f7-b860-61310127dafd None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:22.152841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 658/1295] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 4 14:42:21 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:42:24.432270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c586010-e21a-4ad7-a8d4-b08865d2ed57 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:24.433129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c586010-e21a-4ad7-a8d4-b08865d2ed57 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:24.433129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c586010-e21a-4ad7-a8d4-b08865d2ed57 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:24.434293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 638/1296] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:42:24.440427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820837b9-1ce9-4719-ab90-41f01c3de264 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:24.441091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820837b9-1ce9-4719-ab90-41f01c3de264 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:24.441091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820837b9-1ce9-4719-ab90-41f01c3de264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:24.459221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-820837b9-1ce9-4719-ab90-41f01c3de264 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:42:24.460467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-820837b9-1ce9-4719-ab90-41f01c3de264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:42:24.549091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 659/1297] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:42:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 04 14:42:24.572565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c586010-e21a-4ad7-a8d4-b08865d2ed57 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:24.685155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c586010-e21a-4ad7-a8d4-b08865d2ed57 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:24.723491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:24.724305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:24.724580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:24.724853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:24.725547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:24.726919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:24.739629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:24.741738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:24.743776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 639/1298] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:42:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:24.749125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820837b9-1ce9-4719-ab90-41f01c3de264 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:24.856594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820837b9-1ce9-4719-ab90-41f01c3de264 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:24.891319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:24.891999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:24.892587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:24.892897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:24.893613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] RBAC: Adding query filter params (name=placement) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:24.893910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:24.895286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:24.901112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 660/1299] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 4 14:42:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:42:24.908160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:25.061239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6c4aa-80bd-49f8-86b5-13befcb7b200 None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:25.096093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749120b4-b4a1-4b66-b523-73de71afd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:25.096750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749120b4-b4a1-4b66-b523-73de71afd30a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:25.097055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749120b4-b4a1-4b66-b523-73de71afd30a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:25.097325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749120b4-b4a1-4b66-b523-73de71afd30a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:25.098121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749120b4-b4a1-4b66-b523-73de71afd30a None admin] RBAC: Adding query filter params (interface=public, service_id=e223c88c489a46228d8023152bc9134a, region_id=RegionOne) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:25.098426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749120b4-b4a1-4b66-b523-73de71afd30a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:25.099767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749120b4-b4a1-4b66-b523-73de71afd30a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:25.105438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 640/1300] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 4 14:42:24 2022] GET /identity/v3/endpoints?service_id=e223c88c489a46228d8023152bc9134a&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:25.111710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:25.226350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c035c-4c1d-418a-a81d-3b56b1d287bc None admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:25.261989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4340fe-a409-4fde-953a-ed86c96662a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:25.262671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4340fe-a409-4fde-953a-ed86c96662a6 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:25.262946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4340fe-a409-4fde-953a-ed86c96662a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:25.263221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4340fe-a409-4fde-953a-ed86c96662a6 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:25.263927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4340fe-a409-4fde-953a-ed86c96662a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:25.265458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4340fe-a409-4fde-953a-ed86c96662a6 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:25.273675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 661/1301] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 4 14:42:25 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:42:27.510120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a2fea-12e7-4e6e-82a0-c95310dfae99 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:27.510839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a2fea-12e7-4e6e-82a0-c95310dfae99 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:27.510839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a2fea-12e7-4e6e-82a0-c95310dfae99 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:27.512072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 641/1302] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:42:27.518230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615c5db-5959-4ebe-ad35-43d51b6f50e4 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:27.518729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615c5db-5959-4ebe-ad35-43d51b6f50e4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:27.518833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615c5db-5959-4ebe-ad35-43d51b6f50e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:27.536239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d615c5db-5959-4ebe-ad35-43d51b6f50e4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:42:27.537513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d615c5db-5959-4ebe-ad35-43d51b6f50e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:42:27.614919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 662/1303] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:42:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 04 14:42:27.638416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a2fea-12e7-4e6e-82a0-c95310dfae99 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:27.743533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a2fea-12e7-4e6e-82a0-c95310dfae99 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:27.777890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:27.778704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:27.778986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:27.779257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] PATH_INFO: `/v3/services/placement` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:27.779921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:27.781409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application Traceback (most recent call last): May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 04 14:42:27.794309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._get_service(service_id) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 04 14:42:27.796545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.get_or_create( May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application with Lock( May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self._enter() May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application return self.creator() May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application created_value = creator( May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: ERROR keystone.server.flask.application May 04 14:42:27.798499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 642/1304] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 4 14:42:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 04 14:42:27.803936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615c5db-5959-4ebe-ad35-43d51b6f50e4 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:27.909826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615c5db-5959-4ebe-ad35-43d51b6f50e4 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:27.944440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93d48-5984-4869-8d06-b0840dc00a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:27.945133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93d48-5984-4869-8d06-b0840dc00a39 None admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:27.945411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93d48-5984-4869-8d06-b0840dc00a39 None admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:27.945679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93d48-5984-4869-8d06-b0840dc00a39 None admin] PATH_INFO: `/v3/services` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:27.946380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c93d48-5984-4869-8d06-b0840dc00a39 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:42:27.946680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c93d48-5984-4869-8d06-b0840dc00a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:27.948038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c93d48-5984-4869-8d06-b0840dc00a39 None admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:27.953249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 663/1305] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 4 14:42:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:42:27.959968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:42:28.085278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebeda7-a5a7-4a91-8aea-5d60e84a386a None admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:42:28.119158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732f03b-5ada-4666-a795-f186748110ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:42:28.119810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732f03b-5ada-4666-a795-f186748110ca None admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:42:28.120097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732f03b-5ada-4666-a795-f186748110ca None admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:42:28.120370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732f03b-5ada-4666-a795-f186748110ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:42:28.121155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732f03b-5ada-4666-a795-f186748110ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:42:28.122512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732f03b-5ada-4666-a795-f186748110ca None admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:42:28.142048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 643/1306] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 4 14:42:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:43:35.748557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e8f63-51bc-4b90-b9aa-18e3be77120f None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:35.752370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e8f63-51bc-4b90-b9aa-18e3be77120f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:35.754291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e8f63-51bc-4b90-b9aa-18e3be77120f None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:35.765580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 664/1307] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:35 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:35.770468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ddf1d-3468-45ff-a5b9-af5d1407930e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:35.772259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ddf1d-3468-45ff-a5b9-af5d1407930e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:35.773517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ddf1d-3468-45ff-a5b9-af5d1407930e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:35.932899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-9e3ddf1d-3468-45ff-a5b9-af5d1407930e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:35.934549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9e3ddf1d-3468-45ff-a5b9-af5d1407930e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:36.063030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 644/1308] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:36.112251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d34e8-8471-44e8-b615-d2d6e4568bf7 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:36.112612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d34e8-8471-44e8-b615-d2d6e4568bf7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:36.113138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d34e8-8471-44e8-b615-d2d6e4568bf7 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:36.114858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 665/1309] 10.222.0.32 () {54 vars in 964 bytes} [Wed May 4 14:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:36.121133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857525b-5133-4ecc-b003-93769867146e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:36.121418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857525b-5133-4ecc-b003-93769867146e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:36.121695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857525b-5133-4ecc-b003-93769867146e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:36.199846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-2857525b-5133-4ecc-b003-93769867146e None None] MFA Rules not processed for user `0d678d5ee7d74896bbc1316b32ec81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:36.201118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2857525b-5133-4ecc-b003-93769867146e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:36.281545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 645/1310] 10.222.0.32 () {58 vars in 1061 bytes} [Wed May 4 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:36.290463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d34e8-8471-44e8-b615-d2d6e4568bf7 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:36.292180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b19d34e8-8471-44e8-b615-d2d6e4568bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:36.419652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d34e8-8471-44e8-b615-d2d6e4568bf7 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:36.495211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bfa2f-7fb6-45d3-8bce-3734edd8dbc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:36.495999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bfa2f-7fb6-45d3-8bce-3734edd8dbc4 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:36.496279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bfa2f-7fb6-45d3-8bce-3734edd8dbc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:36.496560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bfa2f-7fb6-45d3-8bce-3734edd8dbc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:36.498355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ae3bfa2f-7fb6-45d3-8bce-3734edd8dbc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:36.613503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bfa2f-7fb6-45d3-8bce-3734edd8dbc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:36.615600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bfa2f-7fb6-45d3-8bce-3734edd8dbc4 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:36.636614 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 666/1311] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:43:39.324082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62e1b-065e-4273-ad5e-acf332e23fa5 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:39.324469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62e1b-065e-4273-ad5e-acf332e23fa5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:39.324630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62e1b-065e-4273-ad5e-acf332e23fa5 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:39.327322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 646/1312] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:39.331844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf29a5-a6ed-4a14-bbaf-fde4d09ce55d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:39.332309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf29a5-a6ed-4a14-bbaf-fde4d09ce55d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:39.332589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf29a5-a6ed-4a14-bbaf-fde4d09ce55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:39.414618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2ccf29a5-a6ed-4a14-bbaf-fde4d09ce55d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:39.415985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2ccf29a5-a6ed-4a14-bbaf-fde4d09ce55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:39.513452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 667/1313] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:39.552137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62e1b-065e-4273-ad5e-acf332e23fa5 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:39.553864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-bdb62e1b-065e-4273-ad5e-acf332e23fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:39.669979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62e1b-065e-4273-ad5e-acf332e23fa5 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:39.754605 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e89c5-fc8d-4095-864f-4f0dcea42779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:39.755262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e89c5-fc8d-4095-864f-4f0dcea42779 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:39.755540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e89c5-fc8d-4095-864f-4f0dcea42779 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:39.755809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e89c5-fc8d-4095-864f-4f0dcea42779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:39.757505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d23e89c5-fc8d-4095-864f-4f0dcea42779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:39.874920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e89c5-fc8d-4095-864f-4f0dcea42779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:39.876385 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e89c5-fc8d-4095-864f-4f0dcea42779 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:39.896646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 647/1314] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:43:42.486309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9df87-b495-48e7-8707-2fd49a3a2b58 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:42.486931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9df87-b495-48e7-8707-2fd49a3a2b58 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:42.486931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9df87-b495-48e7-8707-2fd49a3a2b58 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:42.488256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 668/1315] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:42.493579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03b2ce-9399-475d-9f63-f2f12d15c759 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:42.494018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03b2ce-9399-475d-9f63-f2f12d15c759 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:42.494122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03b2ce-9399-475d-9f63-f2f12d15c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:42.560018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-af03b2ce-9399-475d-9f63-f2f12d15c759 None None] MFA Rules not processed for user `9a3eded2657c435894f06d8e9f184826`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:42.561277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-af03b2ce-9399-475d-9f63-f2f12d15c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:42.658847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 648/1316] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:42.699547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9df87-b495-48e7-8707-2fd49a3a2b58 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:42.701258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9be9df87-b495-48e7-8707-2fd49a3a2b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:42.805367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9df87-b495-48e7-8707-2fd49a3a2b58 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:42.884599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8322b-6b1e-45b6-9548-83acec4c01fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:42.885273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8322b-6b1e-45b6-9548-83acec4c01fd service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:42.885555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8322b-6b1e-45b6-9548-83acec4c01fd service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:42.885825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8322b-6b1e-45b6-9548-83acec4c01fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:42.887541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a0d8322b-6b1e-45b6-9548-83acec4c01fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:43.004203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8322b-6b1e-45b6-9548-83acec4c01fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:43.005693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8322b-6b1e-45b6-9548-83acec4c01fd service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:43.024159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 669/1317] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:43:45.848119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972ed-eef6-4832-a7f3-9b2ea0d11faa None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:45.848732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972ed-eef6-4832-a7f3-9b2ea0d11faa None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:45.848732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972ed-eef6-4832-a7f3-9b2ea0d11faa None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:45.850081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 649/1318] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:45.855535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bf3c7-3699-4466-889a-8c8c7b4874a6 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:45.855959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bf3c7-3699-4466-889a-8c8c7b4874a6 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:45.856072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bf3c7-3699-4466-889a-8c8c7b4874a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:45.926537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-049bf3c7-3699-4466-889a-8c8c7b4874a6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:45.927813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-049bf3c7-3699-4466-889a-8c8c7b4874a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:46.027881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 670/1319] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:46.066261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972ed-eef6-4832-a7f3-9b2ea0d11faa None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:46.067826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-878972ed-eef6-4832-a7f3-9b2ea0d11faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:46.175214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972ed-eef6-4832-a7f3-9b2ea0d11faa None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:46.253086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cb745-5135-4325-a36c-6470c7fd69d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:46.253739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cb745-5135-4325-a36c-6470c7fd69d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:46.254089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cb745-5135-4325-a36c-6470c7fd69d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:46.254521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cb745-5135-4325-a36c-6470c7fd69d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:46.256197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-562cb745-5135-4325-a36c-6470c7fd69d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:46.382093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cb745-5135-4325-a36c-6470c7fd69d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:46.383540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cb745-5135-4325-a36c-6470c7fd69d0 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:46.404397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 650/1320] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:43:49.380305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac1cd0-00ca-4624-b66a-b3f7dfa13af2 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:49.380856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac1cd0-00ca-4624-b66a-b3f7dfa13af2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:49.380856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac1cd0-00ca-4624-b66a-b3f7dfa13af2 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:49.382398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 671/1321] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:49.387893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f638c-6437-4146-95e2-e96cd8b57420 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:49.388362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f638c-6437-4146-95e2-e96cd8b57420 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:49.388452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f638c-6437-4146-95e2-e96cd8b57420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:49.457460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a97f638c-6437-4146-95e2-e96cd8b57420 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:49.458665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a97f638c-6437-4146-95e2-e96cd8b57420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:49.554240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 651/1322] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:49.590460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac1cd0-00ca-4624-b66a-b3f7dfa13af2 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:49.592009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-efac1cd0-00ca-4624-b66a-b3f7dfa13af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:49.697553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac1cd0-00ca-4624-b66a-b3f7dfa13af2 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:49.774681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d21c94-9cef-4e84-842f-571c541b174f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:49.775334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21c94-9cef-4e84-842f-571c541b174f service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:49.775589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21c94-9cef-4e84-842f-571c541b174f service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:49.775857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21c94-9cef-4e84-842f-571c541b174f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:49.777721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-64d21c94-9cef-4e84-842f-571c541b174f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:49.892269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d21c94-9cef-4e84-842f-571c541b174f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:49.893703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d21c94-9cef-4e84-842f-571c541b174f service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:49.914273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 672/1323] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:43:52.225736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c3f5d-38e6-454b-80bd-42d93a91d796 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:52.226284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c3f5d-38e6-454b-80bd-42d93a91d796 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:52.226284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c3f5d-38e6-454b-80bd-42d93a91d796 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:52.227825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 652/1324] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:43:52.233791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51c519-8fd3-4efa-ae48-c970225bdd2d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:52.234332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51c519-8fd3-4efa-ae48-c970225bdd2d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:52.234332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51c519-8fd3-4efa-ae48-c970225bdd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:52.300688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6c51c519-8fd3-4efa-ae48-c970225bdd2d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:52.301922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6c51c519-8fd3-4efa-ae48-c970225bdd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:52.404854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 673/1325] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:43:52.456853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea9b2-0e01-4670-8cd4-6a4eedb96bf3 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:52.457144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea9b2-0e01-4670-8cd4-6a4eedb96bf3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:52.457423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea9b2-0e01-4670-8cd4-6a4eedb96bf3 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:52.458851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 653/1326] 10.222.0.32 () {54 vars in 964 bytes} [Wed May 4 14:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:52.465391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09a828-e553-4c38-a01f-847d6e72151e None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:52.465663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09a828-e553-4c38-a01f-847d6e72151e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:52.465970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09a828-e553-4c38-a01f-847d6e72151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:52.543204 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6f09a828-e553-4c38-a01f-847d6e72151e None None] MFA Rules not processed for user `0d678d5ee7d74896bbc1316b32ec81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:52.544452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6f09a828-e553-4c38-a01f-847d6e72151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:52.625326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 674/1327] 10.222.0.32 () {58 vars in 1061 bytes} [Wed May 4 14:43:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:52.634638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fea9b2-0e01-4670-8cd4-6a4eedb96bf3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:52.636156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-37fea9b2-0e01-4670-8cd4-6a4eedb96bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:52.745423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fea9b2-0e01-4670-8cd4-6a4eedb96bf3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:52.822992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea0375-f1f8-4ac6-889d-3510e5380703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:52.823554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea0375-f1f8-4ac6-889d-3510e5380703 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:52.823833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea0375-f1f8-4ac6-889d-3510e5380703 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:52.824103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea0375-f1f8-4ac6-889d-3510e5380703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:52.825784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7aea0375-f1f8-4ac6-889d-3510e5380703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:52.939236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea0375-f1f8-4ac6-889d-3510e5380703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:52.940699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea0375-f1f8-4ac6-889d-3510e5380703 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:52.959291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 654/1328] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:43:55.505686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035746-c8be-4711-96ad-a4e3a95018cb None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:55.506288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035746-c8be-4711-96ad-a4e3a95018cb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:55.506288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035746-c8be-4711-96ad-a4e3a95018cb None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:55.507640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 675/1329] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:55.513294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eb2aa-71b5-4bf1-8719-45f9135274ab None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:55.513780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eb2aa-71b5-4bf1-8719-45f9135274ab None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:55.513892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eb2aa-71b5-4bf1-8719-45f9135274ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:55.581793 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-509eb2aa-71b5-4bf1-8719-45f9135274ab None None] MFA Rules not processed for user `9a3eded2657c435894f06d8e9f184826`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:55.582927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-509eb2aa-71b5-4bf1-8719-45f9135274ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:55.681205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 655/1330] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:55.718871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e035746-c8be-4711-96ad-a4e3a95018cb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:55.720439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1e035746-c8be-4711-96ad-a4e3a95018cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:55.831014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e035746-c8be-4711-96ad-a4e3a95018cb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:55.906698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24112f17-ea70-4131-88b5-35cf220f6131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:55.907278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24112f17-ea70-4131-88b5-35cf220f6131 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:55.907564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24112f17-ea70-4131-88b5-35cf220f6131 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:55.907825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24112f17-ea70-4131-88b5-35cf220f6131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:55.909514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-24112f17-ea70-4131-88b5-35cf220f6131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:56.022397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24112f17-ea70-4131-88b5-35cf220f6131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:56.024349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24112f17-ea70-4131-88b5-35cf220f6131 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:56.042675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 676/1331] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:43:58.735539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549a17-af20-40b7-a375-5ed02a025ebf None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:58.736154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549a17-af20-40b7-a375-5ed02a025ebf None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:58.736154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12549a17-af20-40b7-a375-5ed02a025ebf None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:58.737642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 656/1332] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:43:58.743471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b0c9a-baf5-4587-8c8c-5390b9d30d92 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:58.744069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b0c9a-baf5-4587-8c8c-5390b9d30d92 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:58.744069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b0c9a-baf5-4587-8c8c-5390b9d30d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:58.818007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ea0b0c9a-baf5-4587-8c8c-5390b9d30d92 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:43:58.819250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ea0b0c9a-baf5-4587-8c8c-5390b9d30d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:58.928394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 677/1333] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:43:58.965178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12549a17-af20-40b7-a375-5ed02a025ebf None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:43:58.966861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-12549a17-af20-40b7-a375-5ed02a025ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:59.081065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12549a17-af20-40b7-a375-5ed02a025ebf None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:43:59.157032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7978-52b0-4eea-9f3b-8fda22810d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:43:59.157570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7978-52b0-4eea-9f3b-8fda22810d2e service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:43:59.157852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7978-52b0-4eea-9f3b-8fda22810d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:43:59.158131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7978-52b0-4eea-9f3b-8fda22810d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:43:59.159785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-282c7978-52b0-4eea-9f3b-8fda22810d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:43:59.274317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c7978-52b0-4eea-9f3b-8fda22810d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:43:59.275768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c7978-52b0-4eea-9f3b-8fda22810d2e service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:43:59.293914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 657/1334] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:01.629865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7511d-7bff-4247-9246-1b78862d7cfc None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:01.630420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7511d-7bff-4247-9246-1b78862d7cfc None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:01.630420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7511d-7bff-4247-9246-1b78862d7cfc None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:01.631802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 678/1335] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:01.637787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f448652-1627-4cda-b6bb-e83a7ee9d8c3 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:01.638380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f448652-1627-4cda-b6bb-e83a7ee9d8c3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:01.638519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f448652-1627-4cda-b6bb-e83a7ee9d8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:01.705748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5f448652-1627-4cda-b6bb-e83a7ee9d8c3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:01.706984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5f448652-1627-4cda-b6bb-e83a7ee9d8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:01.805545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 658/1336] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:01.846271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7511d-7bff-4247-9246-1b78862d7cfc None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:01.847874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6fc7511d-7bff-4247-9246-1b78862d7cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:01.960853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7511d-7bff-4247-9246-1b78862d7cfc None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:02.036492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbd3c8-c951-4b96-9cbd-06fa4314ca24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:02.037173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbd3c8-c951-4b96-9cbd-06fa4314ca24 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:02.037451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbd3c8-c951-4b96-9cbd-06fa4314ca24 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:02.037723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbd3c8-c951-4b96-9cbd-06fa4314ca24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:02.039391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7dbbd3c8-c951-4b96-9cbd-06fa4314ca24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:02.153829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbd3c8-c951-4b96-9cbd-06fa4314ca24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:02.155257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbd3c8-c951-4b96-9cbd-06fa4314ca24 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:02.176361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 679/1337] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:04.773379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77a0e4-660f-41f3-93b8-d04b2fb476af None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:04.774030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77a0e4-660f-41f3-93b8-d04b2fb476af None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:04.774030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77a0e4-660f-41f3-93b8-d04b2fb476af None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:04.775515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 659/1338] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:04.782107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb697b1-3d98-42e9-8828-14bbdb00b4f0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:04.782704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb697b1-3d98-42e9-8828-14bbdb00b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:04.782704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb697b1-3d98-42e9-8828-14bbdb00b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:04.850380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-afb697b1-3d98-42e9-8828-14bbdb00b4f0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:04.851622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-afb697b1-3d98-42e9-8828-14bbdb00b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:04.951096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 680/1339] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:04.987062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77a0e4-660f-41f3-93b8-d04b2fb476af None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:04.988531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ae77a0e4-660f-41f3-93b8-d04b2fb476af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:05.094431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae77a0e4-660f-41f3-93b8-d04b2fb476af None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:05.169257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07d0a7-8e91-42a3-92c5-6d30be08e603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:05.169915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07d0a7-8e91-42a3-92c5-6d30be08e603 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:05.170195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07d0a7-8e91-42a3-92c5-6d30be08e603 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:05.171480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07d0a7-8e91-42a3-92c5-6d30be08e603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:05.173161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8f07d0a7-8e91-42a3-92c5-6d30be08e603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:05.287847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07d0a7-8e91-42a3-92c5-6d30be08e603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:05.289358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07d0a7-8e91-42a3-92c5-6d30be08e603 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:05.308229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 660/1340] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:09.656763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e147157-f78d-4cd5-912e-c8cc3085064e None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:09.657400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e147157-f78d-4cd5-912e-c8cc3085064e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:09.657400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e147157-f78d-4cd5-912e-c8cc3085064e None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:09.658720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 681/1341] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:09.664154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcceba-e495-4498-9f15-bfaa0d92973e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:09.664769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcceba-e495-4498-9f15-bfaa0d92973e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:09.664769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcceba-e495-4498-9f15-bfaa0d92973e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:09.736458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-63fcceba-e495-4498-9f15-bfaa0d92973e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:09.737594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-63fcceba-e495-4498-9f15-bfaa0d92973e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:09.842400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 661/1342] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:09.884711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e147157-f78d-4cd5-912e-c8cc3085064e None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:09.887243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9e147157-f78d-4cd5-912e-c8cc3085064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:09.989837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e147157-f78d-4cd5-912e-c8cc3085064e None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:09.995751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1b5d1-0966-43e7-b6dc-5a7b2c412702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:09.996930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1b5d1-0966-43e7-b6dc-5a7b2c412702 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:09.997221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1b5d1-0966-43e7-b6dc-5a7b2c412702 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:09.997499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1b5d1-0966-43e7-b6dc-5a7b2c412702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:10.008456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1b5d1-0966-43e7-b6dc-5a7b2c412702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:10.009938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1b5d1-0966-43e7-b6dc-5a7b2c412702 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:10.023396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 682/1343] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:13.127061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be1be3-707d-482b-95bb-b8f3238d85b8 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:13.127698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be1be3-707d-482b-95bb-b8f3238d85b8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:13.127698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be1be3-707d-482b-95bb-b8f3238d85b8 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:13.129013 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 662/1344] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:13.134498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1249d42-9f33-4df8-9f48-ceaa6c5a8e74 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:13.134971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1249d42-9f33-4df8-9f48-ceaa6c5a8e74 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:13.135085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1249d42-9f33-4df8-9f48-ceaa6c5a8e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:13.164447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e1249d42-9f33-4df8-9f48-ceaa6c5a8e74 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:13.165701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e1249d42-9f33-4df8-9f48-ceaa6c5a8e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:13.246918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 683/1345] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:13.284859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be1be3-707d-482b-95bb-b8f3238d85b8 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:13.297162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be1be3-707d-482b-95bb-b8f3238d85b8 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:13.301471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741145d-6e61-4539-80d1-93d8cc4f5227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:13.302135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741145d-6e61-4539-80d1-93d8cc4f5227 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:13.302394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741145d-6e61-4539-80d1-93d8cc4f5227 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:13.302666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741145d-6e61-4539-80d1-93d8cc4f5227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:13.316460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741145d-6e61-4539-80d1-93d8cc4f5227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:13.318404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741145d-6e61-4539-80d1-93d8cc4f5227 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:13.329128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 663/1346] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:17.893981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56adc55-09f7-454a-8936-1345817edeb7 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:17.894544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56adc55-09f7-454a-8936-1345817edeb7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:17.894544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56adc55-09f7-454a-8936-1345817edeb7 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:17.895822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 684/1347] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:17.901663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9d02-15de-42a1-b2b5-f294cf70c448 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:17.902122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9d02-15de-42a1-b2b5-f294cf70c448 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:17.902375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9d02-15de-42a1-b2b5-f294cf70c448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:17.922039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b2ac9d02-15de-42a1-b2b5-f294cf70c448 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:17.923271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b2ac9d02-15de-42a1-b2b5-f294cf70c448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:17.932820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 664/1348] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:44:17.971138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56adc55-09f7-454a-8936-1345817edeb7 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:17.984205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56adc55-09f7-454a-8936-1345817edeb7 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:17.988320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264c855-8ae4-4919-9c14-05991e6448d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:17.988997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c855-8ae4-4919-9c14-05991e6448d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:17.989280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c855-8ae4-4919-9c14-05991e6448d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:17.989552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c855-8ae4-4919-9c14-05991e6448d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:18.001385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264c855-8ae4-4919-9c14-05991e6448d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:18.002838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264c855-8ae4-4919-9c14-05991e6448d0 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:18.013038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 685/1349] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:23.277205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231e1e-7ba7-4fd1-9d0f-231c638baa5d None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:23.278101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231e1e-7ba7-4fd1-9d0f-231c638baa5d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:23.278574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37231e1e-7ba7-4fd1-9d0f-231c638baa5d None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:23.280239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 665/1350] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:44:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:23.286381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa07d3-4b65-43bc-841d-0e3eee8d6860 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:23.286833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa07d3-4b65-43bc-841d-0e3eee8d6860 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:23.286917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa07d3-4b65-43bc-841d-0e3eee8d6860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:23.433586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d7fa07d3-4b65-43bc-841d-0e3eee8d6860 None None] MFA Rules not processed for user `7ac21bca8da642338571f821ef5ea121`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:23.434862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d7fa07d3-4b65-43bc-841d-0e3eee8d6860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:23.514204 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 686/1351] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:28.021413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcbb97-39e6-4a7c-bb80-7e09fd90478e None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:28.022026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcbb97-39e6-4a7c-bb80-7e09fd90478e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:28.022026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcbb97-39e6-4a7c-bb80-7e09fd90478e None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:28.023684 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 666/1352] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:28.029455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c2659-e26a-46e2-a760-53c89cead90d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:28.030500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c2659-e26a-46e2-a760-53c89cead90d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:28.030983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c2659-e26a-46e2-a760-53c89cead90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:28.054651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-477c2659-e26a-46e2-a760-53c89cead90d None None] MFA Rules not processed for user `7ac21bca8da642338571f821ef5ea121`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:28.057048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-477c2659-e26a-46e2-a760-53c89cead90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:28.068419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 687/1353] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:44:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:29.477379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846d8-f244-46c2-acf8-360de189d7f9 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:29.478377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846d8-f244-46c2-acf8-360de189d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:29.478650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8846d8-f244-46c2-acf8-360de189d7f9 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:29.480135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 667/1354] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:44:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:29.485863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71554ce-cd48-4b86-8af8-17b2b1e83453 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:29.486330 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71554ce-cd48-4b86-8af8-17b2b1e83453 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:29.486439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71554ce-cd48-4b86-8af8-17b2b1e83453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:29.505075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c71554ce-cd48-4b86-8af8-17b2b1e83453 None None] MFA Rules not processed for user `7ac21bca8da642338571f821ef5ea121`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:29.506296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c71554ce-cd48-4b86-8af8-17b2b1e83453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:29.515700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 688/1355] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:33.525165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6ae1-cb79-4aaa-88d6-01d3d701c482 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:33.525793 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6ae1-cb79-4aaa-88d6-01d3d701c482 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:33.525793 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b6ae1-cb79-4aaa-88d6-01d3d701c482 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:33.527117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 668/1356] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:33.532779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eee22-fb3e-4369-acfb-321a399ca000 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:33.533334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eee22-fb3e-4369-acfb-321a399ca000 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:33.533583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eee22-fb3e-4369-acfb-321a399ca000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:33.553044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-488eee22-fb3e-4369-acfb-321a399ca000 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:33.554436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-488eee22-fb3e-4369-acfb-321a399ca000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:33.563233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 689/1357] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:33.611975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61412227-2ebc-4470-a141-7a2dec7d7217 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:33.612257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61412227-2ebc-4470-a141-7a2dec7d7217 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:33.612707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61412227-2ebc-4470-a141-7a2dec7d7217 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:33.614144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 669/1358] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 4 14:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:33.620251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266fc42-bc4e-4b74-a0f9-a95ebac7e944 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:33.620761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266fc42-bc4e-4b74-a0f9-a95ebac7e944 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:33.621345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266fc42-bc4e-4b74-a0f9-a95ebac7e944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:33.685992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c266fc42-bc4e-4b74-a0f9-a95ebac7e944 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:33.687361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c266fc42-bc4e-4b74-a0f9-a95ebac7e944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:33.775804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 690/1359] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 4 14:44:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:33.784784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61412227-2ebc-4470-a141-7a2dec7d7217 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:33.800679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61412227-2ebc-4470-a141-7a2dec7d7217 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:33.808015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6148e5-c9a2-49fd-8deb-2703ab284849 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:33.808668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6148e5-c9a2-49fd-8deb-2703ab284849 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:33.810971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6148e5-c9a2-49fd-8deb-2703ab284849 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:33.810971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6148e5-c9a2-49fd-8deb-2703ab284849 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:33.826541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6148e5-c9a2-49fd-8deb-2703ab284849 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:33.828412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6148e5-c9a2-49fd-8deb-2703ab284849 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:33.839228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 670/1360] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:36.345462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ffe51-f37e-4bb7-94f3-ce12f57e5141 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:36.346064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ffe51-f37e-4bb7-94f3-ce12f57e5141 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:36.346064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ffe51-f37e-4bb7-94f3-ce12f57e5141 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:36.347554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 691/1361] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:44:36.352782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054e2e-b5c8-4413-af73-49865778df0d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:36.353412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054e2e-b5c8-4413-af73-49865778df0d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:36.353412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054e2e-b5c8-4413-af73-49865778df0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:36.372166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-86054e2e-b5c8-4413-af73-49865778df0d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:36.373700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-86054e2e-b5c8-4413-af73-49865778df0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:36.382462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 671/1362] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:44:36.428018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ffe51-f37e-4bb7-94f3-ce12f57e5141 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:36.443931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ffe51-f37e-4bb7-94f3-ce12f57e5141 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:36.450605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f994bbc4-0657-4bcf-ad1c-3db8a256ed62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:36.451256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994bbc4-0657-4bcf-ad1c-3db8a256ed62 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:36.451543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994bbc4-0657-4bcf-ad1c-3db8a256ed62 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:36.451829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f994bbc4-0657-4bcf-ad1c-3db8a256ed62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:36.461036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994bbc4-0657-4bcf-ad1c-3db8a256ed62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:36.462736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f994bbc4-0657-4bcf-ad1c-3db8a256ed62 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:36.473236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 692/1363] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:39.081562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7c7c-7b45-4814-b63c-69b793546603 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:39.082083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7c7c-7b45-4814-b63c-69b793546603 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:39.082083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7c7c-7b45-4814-b63c-69b793546603 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:39.083775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 672/1364] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:39.089787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039fb8d-5b68-426c-97ac-054609d65d38 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:39.090737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039fb8d-5b68-426c-97ac-054609d65d38 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:39.091163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039fb8d-5b68-426c-97ac-054609d65d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:39.109986 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6039fb8d-5b68-426c-97ac-054609d65d38 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:39.111115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6039fb8d-5b68-426c-97ac-054609d65d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:39.119960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 693/1365] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:39.172367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076503-411d-4112-97bf-6fd7840eb73a None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:39.172681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076503-411d-4112-97bf-6fd7840eb73a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:39.173005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076503-411d-4112-97bf-6fd7840eb73a None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:39.174446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 673/1366] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 4 14:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:39.179994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582e03e-ab59-41ee-8e19-cd157b583e92 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:39.180357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582e03e-ab59-41ee-8e19-cd157b583e92 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:39.180700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582e03e-ab59-41ee-8e19-cd157b583e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:39.198775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b582e03e-ab59-41ee-8e19-cd157b583e92 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:39.199959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b582e03e-ab59-41ee-8e19-cd157b583e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:39.209314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 694/1367] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 4 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:44:39.217437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79076503-411d-4112-97bf-6fd7840eb73a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:39.232228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79076503-411d-4112-97bf-6fd7840eb73a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:39.238507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714eaf97-3499-46e1-b223-5ba79ba67b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:39.239186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eaf97-3499-46e1-b223-5ba79ba67b82 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:39.239484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eaf97-3499-46e1-b223-5ba79ba67b82 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:39.239794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714eaf97-3499-46e1-b223-5ba79ba67b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:39.248723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eaf97-3499-46e1-b223-5ba79ba67b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:39.250462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714eaf97-3499-46e1-b223-5ba79ba67b82 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:39.260177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 674/1368] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:41.612392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4214f-a84d-40a8-b93a-f169b3ed346b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:41.613037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4214f-a84d-40a8-b93a-f169b3ed346b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:41.613037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4214f-a84d-40a8-b93a-f169b3ed346b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:41.614329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 695/1369] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:41.620287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188a8fc-6efb-435b-8cee-2b42ddc549dd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:41.620953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188a8fc-6efb-435b-8cee-2b42ddc549dd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:41.620953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188a8fc-6efb-435b-8cee-2b42ddc549dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:41.641284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6188a8fc-6efb-435b-8cee-2b42ddc549dd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:41.642520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6188a8fc-6efb-435b-8cee-2b42ddc549dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:41.651402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 675/1370] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:41.697336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4214f-a84d-40a8-b93a-f169b3ed346b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:41.712398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4214f-a84d-40a8-b93a-f169b3ed346b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:41.718220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99052256-ebfc-4a00-9589-3b245ac301a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:41.719113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99052256-ebfc-4a00-9589-3b245ac301a7 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:41.719542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99052256-ebfc-4a00-9589-3b245ac301a7 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:41.719955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99052256-ebfc-4a00-9589-3b245ac301a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:41.729283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99052256-ebfc-4a00-9589-3b245ac301a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:41.731244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99052256-ebfc-4a00-9589-3b245ac301a7 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:41.744184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 696/1371] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:41.824091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2118-01ed-486d-8438-15bdcb000a30 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:41.824325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2118-01ed-486d-8438-15bdcb000a30 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:41.824518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2118-01ed-486d-8438-15bdcb000a30 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:41.826242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 676/1372] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:41.832985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf416919-c092-4c38-bf2c-771dd6c0acec None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:41.833673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf416919-c092-4c38-bf2c-771dd6c0acec None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:41.834098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf416919-c092-4c38-bf2c-771dd6c0acec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:41.857707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-bf416919-c092-4c38-bf2c-771dd6c0acec None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:41.859305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bf416919-c092-4c38-bf2c-771dd6c0acec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:41.872872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 697/1373] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:42.150586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f162bf-c100-4eba-800c-d38a437eb07e None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:42.150878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f162bf-c100-4eba-800c-d38a437eb07e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:42.151165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f162bf-c100-4eba-800c-d38a437eb07e None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:42.152555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 677/1374] 10.222.0.32 () {54 vars in 963 bytes} [Wed May 4 14:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:42.158073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a8c2-f22b-4b5f-be59-4988a720428b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:42.158361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a8c2-f22b-4b5f-be59-4988a720428b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:42.158839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a8c2-f22b-4b5f-be59-4988a720428b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:42.209236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8396a8c2-f22b-4b5f-be59-4988a720428b None None] MFA Rules not processed for user `a4aed1720aea4ffa975f7936a1bdb3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:42.210443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8396a8c2-f22b-4b5f-be59-4988a720428b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:42.273171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 698/1375] 10.222.0.32 () {58 vars in 1060 bytes} [Wed May 4 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:42.281778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f162bf-c100-4eba-800c-d38a437eb07e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:42.293791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f162bf-c100-4eba-800c-d38a437eb07e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:42.297861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787b504-8a36-43bb-9ecb-477721ed382d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:42.299014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787b504-8a36-43bb-9ecb-477721ed382d service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:42.299280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787b504-8a36-43bb-9ecb-477721ed382d service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:42.299570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787b504-8a36-43bb-9ecb-477721ed382d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:42.314377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787b504-8a36-43bb-9ecb-477721ed382d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:42.316161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787b504-8a36-43bb-9ecb-477721ed382d service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:42.330916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 678/1376] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:44.039150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973dc90-ab7e-4faa-9d01-301265bf7139 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:44.039813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973dc90-ab7e-4faa-9d01-301265bf7139 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:44.039813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973dc90-ab7e-4faa-9d01-301265bf7139 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:44.041277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 699/1377] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:44.047267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b49944-6fbd-4bbe-b314-aa031a9487db None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:44.048073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b49944-6fbd-4bbe-b314-aa031a9487db None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:44.048073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b49944-6fbd-4bbe-b314-aa031a9487db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:44.066495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-24b49944-6fbd-4bbe-b314-aa031a9487db None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:44.067778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-24b49944-6fbd-4bbe-b314-aa031a9487db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:44.076581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 679/1378] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:44.121048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973dc90-ab7e-4faa-9d01-301265bf7139 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:44.136770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973dc90-ab7e-4faa-9d01-301265bf7139 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:44.143134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe9cd5-d9f6-47e2-920e-43261645731c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:44.143798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe9cd5-d9f6-47e2-920e-43261645731c service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:44.144108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe9cd5-d9f6-47e2-920e-43261645731c service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:44.144386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe9cd5-d9f6-47e2-920e-43261645731c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:44.153788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe9cd5-d9f6-47e2-920e-43261645731c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:44.155408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe9cd5-d9f6-47e2-920e-43261645731c service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:44.165486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 700/1379] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:46.506212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06f7f-19bf-46fc-b9db-be51c432f0a5 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:46.506810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06f7f-19bf-46fc-b9db-be51c432f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:46.506810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b06f7f-19bf-46fc-b9db-be51c432f0a5 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:46.508134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 680/1380] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:44:46.514370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6f1d0-30bf-4cfc-b0c9-e100fb2a0833 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:46.514874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6f1d0-30bf-4cfc-b0c9-e100fb2a0833 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:46.514874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6f1d0-30bf-4cfc-b0c9-e100fb2a0833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:46.533693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-baf6f1d0-30bf-4cfc-b0c9-e100fb2a0833 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:46.535018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-baf6f1d0-30bf-4cfc-b0c9-e100fb2a0833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:46.543395 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 701/1381] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:44:46.587629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06f7f-19bf-46fc-b9db-be51c432f0a5 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:46.603348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b06f7f-19bf-46fc-b9db-be51c432f0a5 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:46.609448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9638c1-536b-40b3-b184-ad347b7c0f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:46.610214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9638c1-536b-40b3-b184-ad347b7c0f00 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:46.610572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9638c1-536b-40b3-b184-ad347b7c0f00 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:46.610909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9638c1-536b-40b3-b184-ad347b7c0f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:46.619883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9638c1-536b-40b3-b184-ad347b7c0f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:46.621997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9638c1-536b-40b3-b184-ad347b7c0f00 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:46.632494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 681/1382] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:48.937925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613e94-a3b8-4229-a3ea-f83da9f5aba5 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:48.938839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613e94-a3b8-4229-a3ea-f83da9f5aba5 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:48.939320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613e94-a3b8-4229-a3ea-f83da9f5aba5 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:48.940674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 702/1383] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:48.946204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bcc33-30b9-40bc-b91a-9a0bf973e1e1 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:48.946681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bcc33-30b9-40bc-b91a-9a0bf973e1e1 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:48.946799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bcc33-30b9-40bc-b91a-9a0bf973e1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:48.965252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-fb4bcc33-30b9-40bc-b91a-9a0bf973e1e1 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:48.966434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fb4bcc33-30b9-40bc-b91a-9a0bf973e1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:48.974406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 682/1384] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:49.018083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613e94-a3b8-4229-a3ea-f83da9f5aba5 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:49.033042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613e94-a3b8-4229-a3ea-f83da9f5aba5 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:49.039716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f449df-8f2f-496a-9aee-873c23e2209d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:49.040600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f449df-8f2f-496a-9aee-873c23e2209d service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:49.040942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f449df-8f2f-496a-9aee-873c23e2209d service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:49.041216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f449df-8f2f-496a-9aee-873c23e2209d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:49.050874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f449df-8f2f-496a-9aee-873c23e2209d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:49.052450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f449df-8f2f-496a-9aee-873c23e2209d service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:49.062436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 703/1385] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:51.375554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a905ff-6914-4eb9-aea8-6023f168ea9d None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:51.380847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a905ff-6914-4eb9-aea8-6023f168ea9d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:51.380847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a905ff-6914-4eb9-aea8-6023f168ea9d None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:51.380847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 683/1386] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:51.384591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3712872-2a37-4d92-9e6d-ef740b95cba8 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:51.385234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3712872-2a37-4d92-9e6d-ef740b95cba8 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:51.385544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3712872-2a37-4d92-9e6d-ef740b95cba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:51.408597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f3712872-2a37-4d92-9e6d-ef740b95cba8 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:51.410282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f3712872-2a37-4d92-9e6d-ef740b95cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:51.423177 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 704/1387] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:51.483907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a905ff-6914-4eb9-aea8-6023f168ea9d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:51.500109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a905ff-6914-4eb9-aea8-6023f168ea9d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:51.506651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ec37f-b0f2-4232-b8f5-e0abfd2a1dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:51.507311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec37f-b0f2-4232-b8f5-e0abfd2a1dec service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:51.507610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec37f-b0f2-4232-b8f5-e0abfd2a1dec service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:51.507869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ec37f-b0f2-4232-b8f5-e0abfd2a1dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:51.517762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ec37f-b0f2-4232-b8f5-e0abfd2a1dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:51.519445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ec37f-b0f2-4232-b8f5-e0abfd2a1dec service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:51.529627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 684/1388] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:54.111490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cda90-4ff9-4ad1-955e-bde3c1f3c561 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:54.112122 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cda90-4ff9-4ad1-955e-bde3c1f3c561 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:54.112458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cda90-4ff9-4ad1-955e-bde3c1f3c561 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:54.115073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 705/1389] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:54.123457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22024a25-6956-442b-906f-3942c4c10f67 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:54.123457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22024a25-6956-442b-906f-3942c4c10f67 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:54.123457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22024a25-6956-442b-906f-3942c4c10f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:54.147347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-22024a25-6956-442b-906f-3942c4c10f67 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:54.148969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-22024a25-6956-442b-906f-3942c4c10f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:54.159084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 685/1390] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:54.208694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cda90-4ff9-4ad1-955e-bde3c1f3c561 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:54.240576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cda90-4ff9-4ad1-955e-bde3c1f3c561 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:54.252742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354eb2fc-7a43-487f-a5ec-d85937d4b137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:54.254227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eb2fc-7a43-487f-a5ec-d85937d4b137 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:54.254830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eb2fc-7a43-487f-a5ec-d85937d4b137 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:54.255480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eb2fc-7a43-487f-a5ec-d85937d4b137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:54.269640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354eb2fc-7a43-487f-a5ec-d85937d4b137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:54.272058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354eb2fc-7a43-487f-a5ec-d85937d4b137 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:54.285814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 706/1391] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:56.955960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902168d-1484-47bd-ae8a-2d9a3b83c750 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:56.956610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902168d-1484-47bd-ae8a-2d9a3b83c750 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:56.956610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902168d-1484-47bd-ae8a-2d9a3b83c750 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:56.957964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 686/1392] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:56.964337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770db4-dd2d-414e-a499-21b2be736590 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:56.964915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770db4-dd2d-414e-a499-21b2be736590 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:56.964915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770db4-dd2d-414e-a499-21b2be736590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:56.984850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-9e770db4-dd2d-414e-a499-21b2be736590 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:56.986087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9e770db4-dd2d-414e-a499-21b2be736590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:56.995357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 707/1393] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:57.041137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902168d-1484-47bd-ae8a-2d9a3b83c750 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:57.057100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902168d-1484-47bd-ae8a-2d9a3b83c750 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:57.063190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2888b4-3593-4842-9e9b-105792219c0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:57.063854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2888b4-3593-4842-9e9b-105792219c0e service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:57.064146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2888b4-3593-4842-9e9b-105792219c0e service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:57.064416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2888b4-3593-4842-9e9b-105792219c0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:57.073673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2888b4-3593-4842-9e9b-105792219c0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:57.075257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2888b4-3593-4842-9e9b-105792219c0e service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:57.085765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 687/1394] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:58.535218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b6ad-9272-443e-b4b1-51d3c681f064 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:58.535808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b6ad-9272-443e-b4b1-51d3c681f064 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:58.535808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984b6ad-9272-443e-b4b1-51d3c681f064 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:58.537864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 708/1395] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:58.547582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b979fe-e299-4a0b-a629-adfcfc72ce8c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:58.548190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b979fe-e299-4a0b-a629-adfcfc72ce8c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:58.548521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b979fe-e299-4a0b-a629-adfcfc72ce8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:58.567790 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e4b979fe-e299-4a0b-a629-adfcfc72ce8c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:58.569121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e4b979fe-e299-4a0b-a629-adfcfc72ce8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:58.577887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 688/1396] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:58.634894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b0ca-4ac1-42fb-879a-663a001df2ce None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:58.635325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b0ca-4ac1-42fb-879a-663a001df2ce None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:58.635601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b0ca-4ac1-42fb-879a-663a001df2ce None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:58.637222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 709/1397] 10.222.0.32 () {54 vars in 962 bytes} [Wed May 4 14:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:58.643456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbc128-1a42-4548-b80f-9bea3c49cd28 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:58.643824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbc128-1a42-4548-b80f-9bea3c49cd28 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:58.645211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbc128-1a42-4548-b80f-9bea3c49cd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:58.693381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4fdbc128-1a42-4548-b80f-9bea3c49cd28 None None] MFA Rules not processed for user `fd3eb9e77dc64869b03b1190cf30e54b`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:58.694568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4fdbc128-1a42-4548-b80f-9bea3c49cd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:58.757229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 689/1398] 10.222.0.32 () {58 vars in 1059 bytes} [Wed May 4 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:58.765253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7b0ca-4ac1-42fb-879a-663a001df2ce None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:58.776508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7b0ca-4ac1-42fb-879a-663a001df2ce None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:58.780394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383beffa-dd24-4f30-973d-afbf575be2c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:58.781064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383beffa-dd24-4f30-973d-afbf575be2c7 service glance] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:58.781349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383beffa-dd24-4f30-973d-afbf575be2c7 service glance] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:58.781615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383beffa-dd24-4f30-973d-afbf575be2c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:58.791496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383beffa-dd24-4f30-973d-afbf575be2c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:58.792979 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383beffa-dd24-4f30-973d-afbf575be2c7 service glance] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:58.803337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 710/1399] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:44:59.239098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3edee-1637-4ef9-a12d-361587014806 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:59.239387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3edee-1637-4ef9-a12d-361587014806 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:59.239674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3edee-1637-4ef9-a12d-361587014806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:59.288334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6ec3edee-1637-4ef9-a12d-361587014806 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:59.289550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6ec3edee-1637-4ef9-a12d-361587014806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:59.356436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 690/1400] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:59.382299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bd67d-7f17-4fda-9ed1-1f2b747542fd None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:59.382652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bd67d-7f17-4fda-9ed1-1f2b747542fd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:59.383122 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bd67d-7f17-4fda-9ed1-1f2b747542fd None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:59.384304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 711/1401] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:44:59.390428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1500a-7cd2-41c2-8c67-56c0f7d22c72 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:59.390711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1500a-7cd2-41c2-8c67-56c0f7d22c72 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:59.390982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1500a-7cd2-41c2-8c67-56c0f7d22c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:59.399212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d7e7d-3060-4345-8f35-f8e22be04e90 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:59.399526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d7e7d-3060-4345-8f35-f8e22be04e90 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:59.399844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d7e7d-3060-4345-8f35-f8e22be04e90 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:59.401357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 712/1402] 10.222.0.32 () {54 vars in 942 bytes} [Wed May 4 14:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:44:59.407896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fbb52-c012-459a-b1ee-18db29c46686 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:59.408342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fbb52-c012-459a-b1ee-18db29c46686 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:59.408626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fbb52-c012-459a-b1ee-18db29c46686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:59.410000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b6a1500a-7cd2-41c2-8c67-56c0f7d22c72 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:59.411197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b6a1500a-7cd2-41c2-8c67-56c0f7d22c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:59.419537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 691/1403] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:44:59.456363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-eb5fbb52-c012-459a-b1ee-18db29c46686 None None] MFA Rules not processed for user `d4ed1c88f3cd478aafc83f4b76d67f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:44:59.457672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-eb5fbb52-c012-459a-b1ee-18db29c46686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:44:59.463877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1500a-7cd2-41c2-8c67-56c0f7d22c72 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:59.476193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1500a-7cd2-41c2-8c67-56c0f7d22c72 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:59.481962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:59.482646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:59.482927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:59.483205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:59.491910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:59.493535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:59.503607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 692/1404] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:44:59.529890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 713/1405] 10.222.0.32 () {58 vars in 1039 bytes} [Wed May 4 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:44:59.542851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:44:59.553441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b04070-c5bc-482c-be81-a90a03ba1672 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:44:59.559341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea6e6a-562f-4449-bc12-5796bfae99bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:44:59.559983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea6e6a-562f-4449-bc12-5796bfae99bc service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:44:59.560262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea6e6a-562f-4449-bc12-5796bfae99bc service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:44:59.560571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea6e6a-562f-4449-bc12-5796bfae99bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:44:59.569898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea6e6a-562f-4449-bc12-5796bfae99bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:44:59.571539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea6e6a-562f-4449-bc12-5796bfae99bc service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:44:59.581232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 693/1406] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:44:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:45:01.878906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25762291-88a8-4ff2-9728-9b365f69b044 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:01.879484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25762291-88a8-4ff2-9728-9b365f69b044 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:01.879484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25762291-88a8-4ff2-9728-9b365f69b044 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:01.881874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 714/1407] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:01.886262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2bf1c-944b-4d8b-92b0-03fec5f61a00 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:01.887093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2bf1c-944b-4d8b-92b0-03fec5f61a00 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:01.887516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2bf1c-944b-4d8b-92b0-03fec5f61a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:01.906189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4ae2bf1c-944b-4d8b-92b0-03fec5f61a00 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:01.907431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4ae2bf1c-944b-4d8b-92b0-03fec5f61a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:01.919860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 694/1408] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:01.960780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25762291-88a8-4ff2-9728-9b365f69b044 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:01.976826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25762291-88a8-4ff2-9728-9b365f69b044 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:01.987586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bccd7e6-e6c0-4ca8-b7bc-2a2202478a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:01.988288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccd7e6-e6c0-4ca8-b7bc-2a2202478a69 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:01.988567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccd7e6-e6c0-4ca8-b7bc-2a2202478a69 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:01.988880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bccd7e6-e6c0-4ca8-b7bc-2a2202478a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:02.000490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccd7e6-e6c0-4ca8-b7bc-2a2202478a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:02.002387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bccd7e6-e6c0-4ca8-b7bc-2a2202478a69 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:02.013879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 715/1409] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:45:22.960925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb49ce9-051b-4bda-b1fd-85adc7ca9db0 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:22.961610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb49ce9-051b-4bda-b1fd-85adc7ca9db0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:22.961610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb49ce9-051b-4bda-b1fd-85adc7ca9db0 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:22.962845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 695/1410] 10.222.0.32 () {54 vars in 918 bytes} [Wed May 4 14:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:45:22.968958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f21673-3af7-46e8-9ef3-b45d60ddcaca None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:22.969542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f21673-3af7-46e8-9ef3-b45d60ddcaca None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:22.969542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f21673-3af7-46e8-9ef3-b45d60ddcaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:22.988216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-75f21673-3af7-46e8-9ef3-b45d60ddcaca None None] MFA Rules not processed for user `7ac21bca8da642338571f821ef5ea121`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:22.989471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-75f21673-3af7-46e8-9ef3-b45d60ddcaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:22.998519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 716/1411] 10.222.0.32 () {58 vars in 1015 bytes} [Wed May 4 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:45:23.014092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365a61b-4df3-4211-9786-78bc07a7ba0f None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:23.014379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365a61b-4df3-4211-9786-78bc07a7ba0f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:23.014658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365a61b-4df3-4211-9786-78bc07a7ba0f None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:23.016435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 696/1412] 10.222.0.32 () {54 vars in 918 bytes} [Wed May 4 14:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:23.021567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be5ca7-e3a8-495d-9d95-3177dc03cd26 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:23.021854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be5ca7-e3a8-495d-9d95-3177dc03cd26 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:23.022127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be5ca7-e3a8-495d-9d95-3177dc03cd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:23.040432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-22be5ca7-e3a8-495d-9d95-3177dc03cd26 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:23.041794 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-22be5ca7-e3a8-495d-9d95-3177dc03cd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:23.051206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 717/1413] 10.222.0.32 () {58 vars in 1015 bytes} [Wed May 4 14:45:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:25.784408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d187d8-a87d-4ebf-aa1a-37d3af0d151a None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:25.785039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d187d8-a87d-4ebf-aa1a-37d3af0d151a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:25.785039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d187d8-a87d-4ebf-aa1a-37d3af0d151a None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:25.786390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:25.791959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d38dd-ada1-40ac-891d-ebe3829609fb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:25.792428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d38dd-ada1-40ac-891d-ebe3829609fb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:25.792532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d38dd-ada1-40ac-891d-ebe3829609fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:25.812301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-611d38dd-ada1-40ac-891d-ebe3829609fb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:25.813571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-611d38dd-ada1-40ac-891d-ebe3829609fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:25.822555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:25.864637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d187d8-a87d-4ebf-aa1a-37d3af0d151a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:25.880028 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d187d8-a87d-4ebf-aa1a-37d3af0d151a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:25.887146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badafa19-2f97-41bd-8316-56981c58e7be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:25.887784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badafa19-2f97-41bd-8316-56981c58e7be service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:25.888063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badafa19-2f97-41bd-8316-56981c58e7be service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:25.888331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badafa19-2f97-41bd-8316-56981c58e7be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:25.897698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badafa19-2f97-41bd-8316-56981c58e7be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:25.899289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badafa19-2f97-41bd-8316-56981c58e7be service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:25.909072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 698/1416] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:27.215180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb264d52-c920-4517-9af8-b477d059577c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:27.215819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb264d52-c920-4517-9af8-b477d059577c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:27.215819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb264d52-c920-4517-9af8-b477d059577c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:27.217148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:27.222187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c9e7e-fdb4-46c4-90fa-a6e15b6a0092 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:27.222634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c9e7e-fdb4-46c4-90fa-a6e15b6a0092 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:27.222740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c9e7e-fdb4-46c4-90fa-a6e15b6a0092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:27.242326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-100c9e7e-fdb4-46c4-90fa-a6e15b6a0092 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:27.243538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-100c9e7e-fdb4-46c4-90fa-a6e15b6a0092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:27.253160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:27.413279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb264d52-c920-4517-9af8-b477d059577c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:27.428560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb264d52-c920-4517-9af8-b477d059577c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:27.434550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733f7a0-c127-4e55-ae52-094340752273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:27.435183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733f7a0-c127-4e55-ae52-094340752273 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:27.435462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733f7a0-c127-4e55-ae52-094340752273 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:27.435734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733f7a0-c127-4e55-ae52-094340752273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:27.444164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733f7a0-c127-4e55-ae52-094340752273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:27.445800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733f7a0-c127-4e55-ae52-094340752273 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:27.455611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 720/1419] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:28.715703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a6812-17ff-4856-aa49-6fd5c20c2924 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:28.716263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a6812-17ff-4856-aa49-6fd5c20c2924 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:28.716263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a6812-17ff-4856-aa49-6fd5c20c2924 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:28.717637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:28.722988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78620f2-665b-4cec-9327-be0174cca107 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:28.723434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78620f2-665b-4cec-9327-be0174cca107 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:28.723527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78620f2-665b-4cec-9327-be0174cca107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:28.742348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e78620f2-665b-4cec-9327-be0174cca107 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:28.743586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e78620f2-665b-4cec-9327-be0174cca107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:28.751740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:28.899861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a6812-17ff-4856-aa49-6fd5c20c2924 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:28.916017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a6812-17ff-4856-aa49-6fd5c20c2924 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:28.922042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7d71-73ee-42f0-b2a3-4aad21726615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:28.922712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7d71-73ee-42f0-b2a3-4aad21726615 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:28.922990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7d71-73ee-42f0-b2a3-4aad21726615 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:28.923260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7d71-73ee-42f0-b2a3-4aad21726615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:28.932753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7d71-73ee-42f0-b2a3-4aad21726615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:28.934392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7d71-73ee-42f0-b2a3-4aad21726615 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:28.945226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 701/1422] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:30.189668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde34f-9696-44bb-b8d8-7d3df2200f5c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:30.190311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde34f-9696-44bb-b8d8-7d3df2200f5c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:30.190311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde34f-9696-44bb-b8d8-7d3df2200f5c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:30.191732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:30.197128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29730569-a914-4ba9-aa1c-b58e99f0e424 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:30.197952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29730569-a914-4ba9-aa1c-b58e99f0e424 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:30.197952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29730569-a914-4ba9-aa1c-b58e99f0e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:30.216715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-29730569-a914-4ba9-aa1c-b58e99f0e424 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:30.217962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-29730569-a914-4ba9-aa1c-b58e99f0e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:30.226437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:30.378668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dde34f-9696-44bb-b8d8-7d3df2200f5c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:30.396353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dde34f-9696-44bb-b8d8-7d3df2200f5c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:30.402632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82763c82-5806-4c5c-84f1-26efed283013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:30.403273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82763c82-5806-4c5c-84f1-26efed283013 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:30.403558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82763c82-5806-4c5c-84f1-26efed283013 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:30.403838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82763c82-5806-4c5c-84f1-26efed283013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:30.415570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82763c82-5806-4c5c-84f1-26efed283013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:30.415570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82763c82-5806-4c5c-84f1-26efed283013 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:30.424336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 723/1425] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:31.675801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85980ea0-8365-4dd4-87fa-7c6681ed1af3 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:31.676379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85980ea0-8365-4dd4-87fa-7c6681ed1af3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:31.676379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85980ea0-8365-4dd4-87fa-7c6681ed1af3 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:31.677838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:31.683452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f057-462f-4b21-a534-d9001045d1de None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:31.684016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f057-462f-4b21-a534-d9001045d1de None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:31.684016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f057-462f-4b21-a534-d9001045d1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:31.703309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d571f057-462f-4b21-a534-d9001045d1de None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:31.704518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d571f057-462f-4b21-a534-d9001045d1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:31.713051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:31.866137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85980ea0-8365-4dd4-87fa-7c6681ed1af3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:31.880746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85980ea0-8365-4dd4-87fa-7c6681ed1af3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:31.887091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3564af1-aea4-493f-ab37-5014739b48c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:31.887741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3564af1-aea4-493f-ab37-5014739b48c8 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:31.888018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3564af1-aea4-493f-ab37-5014739b48c8 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:31.888287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3564af1-aea4-493f-ab37-5014739b48c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:31.897218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3564af1-aea4-493f-ab37-5014739b48c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:31.898800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3564af1-aea4-493f-ab37-5014739b48c8 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:31.908192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 704/1428] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:33.096983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23fbe-c427-4125-9ed6-63b1417f0395 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:33.097720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23fbe-c427-4125-9ed6-63b1417f0395 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:33.097720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23fbe-c427-4125-9ed6-63b1417f0395 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:33.098786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:33.104090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fefe16-9760-415b-828c-1740bc8c83ac None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:33.104581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fefe16-9760-415b-828c-1740bc8c83ac None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:33.104581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fefe16-9760-415b-828c-1740bc8c83ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:33.124227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-43fefe16-9760-415b-828c-1740bc8c83ac None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:33.125457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-43fefe16-9760-415b-828c-1740bc8c83ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:33.135241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:33.283332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23fbe-c427-4125-9ed6-63b1417f0395 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:33.298281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23fbe-c427-4125-9ed6-63b1417f0395 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:33.303995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7e0d8-2d25-48ee-9506-422442c55867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:33.304631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e0d8-2d25-48ee-9506-422442c55867 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:33.304939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e0d8-2d25-48ee-9506-422442c55867 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:33.305214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e0d8-2d25-48ee-9506-422442c55867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:33.314248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7e0d8-2d25-48ee-9506-422442c55867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:33.315921 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7e0d8-2d25-48ee-9506-422442c55867 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:33.327735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 726/1431] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:34.520405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ec6ca-aaef-4337-a673-52b0efb09655 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:34.521062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ec6ca-aaef-4337-a673-52b0efb09655 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:34.521062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ec6ca-aaef-4337-a673-52b0efb09655 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:34.522371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:45:34.527746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee000-63b5-419e-b419-caf786418634 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:34.528248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee000-63b5-419e-b419-caf786418634 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:34.528354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee000-63b5-419e-b419-caf786418634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:34.548493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d0dee000-63b5-419e-b419-caf786418634 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:34.549751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d0dee000-63b5-419e-b419-caf786418634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:34.558977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:34.708176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ec6ca-aaef-4337-a673-52b0efb09655 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:34.725011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ec6ca-aaef-4337-a673-52b0efb09655 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:34.731366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468e213-635f-4364-a03f-bc3e1cf3c4ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:34.732134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468e213-635f-4364-a03f-bc3e1cf3c4ea service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:34.732426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468e213-635f-4364-a03f-bc3e1cf3c4ea service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:34.732754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468e213-635f-4364-a03f-bc3e1cf3c4ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:34.741757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468e213-635f-4364-a03f-bc3e1cf3c4ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:34.743348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468e213-635f-4364-a03f-bc3e1cf3c4ea service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:34.753493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 707/1434] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:35.944033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f123-b6fc-4fc8-98b9-c9c337a3ed95 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:35.944640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f123-b6fc-4fc8-98b9-c9c337a3ed95 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:35.944640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840f123-b6fc-4fc8-98b9-c9c337a3ed95 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:35.946289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:35.952194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a984a-0a1d-4a0c-80bc-3826c7778a43 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:35.952711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a984a-0a1d-4a0c-80bc-3826c7778a43 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:35.952711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a984a-0a1d-4a0c-80bc-3826c7778a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:35.971977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-603a984a-0a1d-4a0c-80bc-3826c7778a43 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:35.973235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-603a984a-0a1d-4a0c-80bc-3826c7778a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:35.981612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:36.128591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f123-b6fc-4fc8-98b9-c9c337a3ed95 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:36.145792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840f123-b6fc-4fc8-98b9-c9c337a3ed95 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:36.151673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cbf41-db3b-4cd9-82b7-0cd085990ca1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:36.152317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cbf41-db3b-4cd9-82b7-0cd085990ca1 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:36.152595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cbf41-db3b-4cd9-82b7-0cd085990ca1 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:36.152895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cbf41-db3b-4cd9-82b7-0cd085990ca1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:36.161397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cbf41-db3b-4cd9-82b7-0cd085990ca1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:36.162966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cbf41-db3b-4cd9-82b7-0cd085990ca1 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:36.172623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 729/1437] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:37.383290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538516-5f95-448d-aca1-b145db5e1540 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:37.383872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538516-5f95-448d-aca1-b145db5e1540 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:37.383872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538516-5f95-448d-aca1-b145db5e1540 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:37.385403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:37.391905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65fc85-1759-4fc4-8f36-81106ed51d87 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:37.392349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65fc85-1759-4fc4-8f36-81106ed51d87 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:37.392451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65fc85-1759-4fc4-8f36-81106ed51d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:37.411651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8d65fc85-1759-4fc4-8f36-81106ed51d87 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:37.412990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8d65fc85-1759-4fc4-8f36-81106ed51d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:37.423021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:37.571423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e538516-5f95-448d-aca1-b145db5e1540 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:37.587679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e538516-5f95-448d-aca1-b145db5e1540 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:37.594165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce0ec4-56fc-41e2-895e-b0c0fbf84bae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:37.594840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce0ec4-56fc-41e2-895e-b0c0fbf84bae service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:37.595118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce0ec4-56fc-41e2-895e-b0c0fbf84bae service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:37.595396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce0ec4-56fc-41e2-895e-b0c0fbf84bae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:37.604983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce0ec4-56fc-41e2-895e-b0c0fbf84bae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:37.606602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce0ec4-56fc-41e2-895e-b0c0fbf84bae service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:37.617510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 710/1440] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:39.782167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c78e1-086b-4582-9d2e-835eb4f171fa None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:39.782785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c78e1-086b-4582-9d2e-835eb4f171fa None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:39.782785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c78e1-086b-4582-9d2e-835eb4f171fa None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:39.784170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:39.789884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d79855-a30c-4aec-bd82-379465c51c85 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:39.790365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d79855-a30c-4aec-bd82-379465c51c85 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:39.790509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d79855-a30c-4aec-bd82-379465c51c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:39.811109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-62d79855-a30c-4aec-bd82-379465c51c85 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:39.812423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-62d79855-a30c-4aec-bd82-379465c51c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:39.822345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:39.862336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c78e1-086b-4582-9d2e-835eb4f171fa None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:39.864465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ee6c78e1-086b-4582-9d2e-835eb4f171fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:39.881922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c78e1-086b-4582-9d2e-835eb4f171fa None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:39.886575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2d96d-ae22-41c1-a507-2510d1acef3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:39.887325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d96d-ae22-41c1-a507-2510d1acef3b service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:39.887596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d96d-ae22-41c1-a507-2510d1acef3b service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:39.887894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d96d-ae22-41c1-a507-2510d1acef3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:39.899575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2d96d-ae22-41c1-a507-2510d1acef3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:39.901341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2d96d-ae22-41c1-a507-2510d1acef3b service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:39.911955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 732/1443] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:42.417385 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cfc7a-e150-4710-99d4-0a02b98ecd89 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:42.417979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cfc7a-e150-4710-99d4-0a02b98ecd89 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:42.417979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cfc7a-e150-4710-99d4-0a02b98ecd89 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:42.419333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:42.424931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb4cc-ed86-4d88-bd11-711fefabc1a0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:42.425413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb4cc-ed86-4d88-bd11-711fefabc1a0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:42.425515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb4cc-ed86-4d88-bd11-711fefabc1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:42.444244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-fc7bb4cc-ed86-4d88-bd11-711fefabc1a0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:42.445487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fc7bb4cc-ed86-4d88-bd11-711fefabc1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:42.454604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:42.498663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cfc7a-e150-4710-99d4-0a02b98ecd89 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:42.511311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cfc7a-e150-4710-99d4-0a02b98ecd89 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:42.517205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94227d0e-22f2-473d-9986-565a19a58ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:42.517897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94227d0e-22f2-473d-9986-565a19a58ecd service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:42.518129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94227d0e-22f2-473d-9986-565a19a58ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:42.518429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94227d0e-22f2-473d-9986-565a19a58ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:42.533222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94227d0e-22f2-473d-9986-565a19a58ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:42.534993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94227d0e-22f2-473d-9986-565a19a58ecd service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:42.545017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 713/1446] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:45.137391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418764-93c2-4d47-a75b-5d509b421755 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:45.138043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418764-93c2-4d47-a75b-5d509b421755 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:45.138043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418764-93c2-4d47-a75b-5d509b421755 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:45.139322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:45.145917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42b83d-5d8d-4028-80da-5eed28b6cb5d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:45.146390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42b83d-5d8d-4028-80da-5eed28b6cb5d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:45.146495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42b83d-5d8d-4028-80da-5eed28b6cb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:45.165781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-cc42b83d-5d8d-4028-80da-5eed28b6cb5d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:45.167008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cc42b83d-5d8d-4028-80da-5eed28b6cb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:45.176578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:45.216246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418764-93c2-4d47-a75b-5d509b421755 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:45.228722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418764-93c2-4d47-a75b-5d509b421755 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:45.234105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b328dcb-8b62-48b3-b524-a52abe9b8777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:45.234751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b328dcb-8b62-48b3-b524-a52abe9b8777 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:45.235028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b328dcb-8b62-48b3-b524-a52abe9b8777 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:45.235295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b328dcb-8b62-48b3-b524-a52abe9b8777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:45.245140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b328dcb-8b62-48b3-b524-a52abe9b8777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:45.246568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b328dcb-8b62-48b3-b524-a52abe9b8777 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:45.256202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 735/1449] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:47.646588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c69d-0d7c-4122-b884-ade5a2cc0bfe None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:47.647153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c69d-0d7c-4122-b884-ade5a2cc0bfe None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:47.647153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45c69d-0d7c-4122-b884-ade5a2cc0bfe None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:47.648526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:47.653971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63308fc-d566-44c8-9c24-7627866211ca None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:47.654446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63308fc-d566-44c8-9c24-7627866211ca None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:47.654555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63308fc-d566-44c8-9c24-7627866211ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:47.673236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a63308fc-d566-44c8-9c24-7627866211ca None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:47.674463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a63308fc-d566-44c8-9c24-7627866211ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:47.683928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:47.732453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45c69d-0d7c-4122-b884-ade5a2cc0bfe None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:47.747687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45c69d-0d7c-4122-b884-ade5a2cc0bfe None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:47.751441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ccfd0d-15d1-43a1-9804-a3d8e8408491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:47.752081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccfd0d-15d1-43a1-9804-a3d8e8408491 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:47.752359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccfd0d-15d1-43a1-9804-a3d8e8408491 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:47.752632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ccfd0d-15d1-43a1-9804-a3d8e8408491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:47.762772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccfd0d-15d1-43a1-9804-a3d8e8408491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:47.764208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ccfd0d-15d1-43a1-9804-a3d8e8408491 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:47.774513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 716/1452] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:50.233725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f629d0d-a8df-4d70-8078-2755bc7073c3 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:50.234324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f629d0d-a8df-4d70-8078-2755bc7073c3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:50.234324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f629d0d-a8df-4d70-8078-2755bc7073c3 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:50.235895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:50.241627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bee7-49fb-4732-8de1-6f18b9a73f6f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:50.242135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bee7-49fb-4732-8de1-6f18b9a73f6f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:50.242135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bee7-49fb-4732-8de1-6f18b9a73f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:50.261462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-90b6bee7-49fb-4732-8de1-6f18b9a73f6f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:50.262715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-90b6bee7-49fb-4732-8de1-6f18b9a73f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:50.271880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:50.316231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f629d0d-a8df-4d70-8078-2755bc7073c3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:50.329157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f629d0d-a8df-4d70-8078-2755bc7073c3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:50.333500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2de6d-02b4-4af0-b964-75ae8b74aeac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:50.334176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2de6d-02b4-4af0-b964-75ae8b74aeac service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:50.334452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2de6d-02b4-4af0-b964-75ae8b74aeac service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:50.334725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2de6d-02b4-4af0-b964-75ae8b74aeac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:50.345873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2de6d-02b4-4af0-b964-75ae8b74aeac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:50.347416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2de6d-02b4-4af0-b964-75ae8b74aeac service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:50.357042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 738/1455] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:52.663003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e046c-c96c-4bfe-807b-bb271b8011a3 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:52.663566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e046c-c96c-4bfe-807b-bb271b8011a3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:52.663566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e046c-c96c-4bfe-807b-bb271b8011a3 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:52.665018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:52.670451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941652b-390c-4ee6-9904-3df648a33887 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:52.670914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941652b-390c-4ee6-9904-3df648a33887 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:52.671010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941652b-390c-4ee6-9904-3df648a33887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:52.689340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-9941652b-390c-4ee6-9904-3df648a33887 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:52.690520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9941652b-390c-4ee6-9904-3df648a33887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:52.698840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:52.738910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e046c-c96c-4bfe-807b-bb271b8011a3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:52.751263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e046c-c96c-4bfe-807b-bb271b8011a3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:52.755011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff907f-8115-4743-8611-ac2e498ad854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:52.755671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff907f-8115-4743-8611-ac2e498ad854 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:52.755957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff907f-8115-4743-8611-ac2e498ad854 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:52.756226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff907f-8115-4743-8611-ac2e498ad854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:52.765984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff907f-8115-4743-8611-ac2e498ad854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:52.767403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff907f-8115-4743-8611-ac2e498ad854 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:52.777289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 719/1458] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:55.103478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe29488-f022-495a-8fe2-59d46f80d444 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:55.104129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe29488-f022-495a-8fe2-59d46f80d444 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:55.104129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe29488-f022-495a-8fe2-59d46f80d444 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:55.105508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:55.112202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d1734-a0ef-4641-85ea-3f875346f7cc None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:55.112973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d1734-a0ef-4641-85ea-3f875346f7cc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:55.113148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d1734-a0ef-4641-85ea-3f875346f7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:55.134574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a86d1734-a0ef-4641-85ea-3f875346f7cc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:55.135859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a86d1734-a0ef-4641-85ea-3f875346f7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:55.146138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:55.192905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe29488-f022-495a-8fe2-59d46f80d444 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:55.204497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe29488-f022-495a-8fe2-59d46f80d444 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:55.208285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a38b-2a23-4f64-8862-035df17f7aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:55.208953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a38b-2a23-4f64-8862-035df17f7aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:55.209234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a38b-2a23-4f64-8862-035df17f7aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:55.209512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a38b-2a23-4f64-8862-035df17f7aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:55.218582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a38b-2a23-4f64-8862-035df17f7aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:55.220009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a38b-2a23-4f64-8862-035df17f7aa9 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:55.229458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 741/1461] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:57.540856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1c273-924e-4045-a6a6-61819e462e3e None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:57.541516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1c273-924e-4045-a6a6-61819e462e3e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:57.541516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1c273-924e-4045-a6a6-61819e462e3e None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:57.542941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:57.549526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c76c-1fee-4bac-bd34-b20225478eb4 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:57.550323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c76c-1fee-4bac-bd34-b20225478eb4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:57.550498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c76c-1fee-4bac-bd34-b20225478eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:57.577713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0e96c76c-1fee-4bac-bd34-b20225478eb4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:57.578842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0e96c76c-1fee-4bac-bd34-b20225478eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:57.587669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:45:57.624868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1c273-924e-4045-a6a6-61819e462e3e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:45:57.636935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1c273-924e-4045-a6a6-61819e462e3e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:45:57.641181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44b471-be9e-4a78-bf88-8791d1b65729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:45:57.641831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44b471-be9e-4a78-bf88-8791d1b65729 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:57.642113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44b471-be9e-4a78-bf88-8791d1b65729 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:57.642383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44b471-be9e-4a78-bf88-8791d1b65729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:57.652465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44b471-be9e-4a78-bf88-8791d1b65729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:45:57.653951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44b471-be9e-4a78-bf88-8791d1b65729 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:45:57.663487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 722/1464] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:45:59.927927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e8059-f90d-4c13-9d88-2622625f9b87 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:59.928498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e8059-f90d-4c13-9d88-2622625f9b87 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:59.928498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e8059-f90d-4c13-9d88-2622625f9b87 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:59.929910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:45:59.936154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22edd-2e99-4293-b402-6771136730f9 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:45:59.936649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22edd-2e99-4293-b402-6771136730f9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:45:59.936767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f22edd-2e99-4293-b402-6771136730f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:45:59.955899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-f8f22edd-2e99-4293-b402-6771136730f9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:45:59.957170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f8f22edd-2e99-4293-b402-6771136730f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:45:59.965462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:46:00.002318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e8059-f90d-4c13-9d88-2622625f9b87 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:00.014460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e8059-f90d-4c13-9d88-2622625f9b87 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:00.018852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b197b-76bd-4d7a-adfb-dc091b821929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:00.019504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b197b-76bd-4d7a-adfb-dc091b821929 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:00.019780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b197b-76bd-4d7a-adfb-dc091b821929 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:00.020051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b197b-76bd-4d7a-adfb-dc091b821929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:00.029554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b197b-76bd-4d7a-adfb-dc091b821929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:00.031000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b197b-76bd-4d7a-adfb-dc091b821929 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:00.040320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 744/1467] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:46:02.313583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e7783-bb8d-4abb-be70-f4a25c7f12da None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:02.314216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e7783-bb8d-4abb-be70-f4a25c7f12da None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:02.314216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e7783-bb8d-4abb-be70-f4a25c7f12da None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:02.315527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:02.321750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6989ede-00f1-4227-8849-a9be001d6eeb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:02.322302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6989ede-00f1-4227-8849-a9be001d6eeb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:02.322302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6989ede-00f1-4227-8849-a9be001d6eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:02.340494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b6989ede-00f1-4227-8849-a9be001d6eeb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:02.341774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b6989ede-00f1-4227-8849-a9be001d6eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:02.350937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:02.387876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e7783-bb8d-4abb-be70-f4a25c7f12da None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:02.401036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e7783-bb8d-4abb-be70-f4a25c7f12da None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:02.405876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03cd85-2081-4a99-a27e-169c448513aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:02.406530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03cd85-2081-4a99-a27e-169c448513aa service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:02.406785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03cd85-2081-4a99-a27e-169c448513aa service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:02.407192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03cd85-2081-4a99-a27e-169c448513aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:02.417562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03cd85-2081-4a99-a27e-169c448513aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:02.419006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03cd85-2081-4a99-a27e-169c448513aa service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:02.428992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 725/1470] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:46:35.749511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917298-6ac8-4027-9ef3-532c405a58da None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:35.750164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917298-6ac8-4027-9ef3-532c405a58da None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:35.750164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917298-6ac8-4027-9ef3-532c405a58da None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:35.751429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 746/1471] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:35.756823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e39e3e-c3ba-4976-a7f1-d442fa2db34c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:35.757306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e39e3e-c3ba-4976-a7f1-d442fa2db34c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:35.757409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e39e3e-c3ba-4976-a7f1-d442fa2db34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:35.777703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d6e39e3e-c3ba-4976-a7f1-d442fa2db34c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:35.778928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d6e39e3e-c3ba-4976-a7f1-d442fa2db34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:35.788619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 726/1472] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:35.837523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d6533-b07b-4748-b95d-54eb15d49053 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:35.837815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d6533-b07b-4748-b95d-54eb15d49053 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:35.838111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d6533-b07b-4748-b95d-54eb15d49053 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:35.839529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 747/1473] 10.222.0.32 () {54 vars in 962 bytes} [Wed May 4 14:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:35.844586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e4ae0-9571-480e-bd3b-6ba8cfc4228c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:35.844958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e4ae0-9571-480e-bd3b-6ba8cfc4228c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:35.845238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e4ae0-9571-480e-bd3b-6ba8cfc4228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:35.864390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d23e4ae0-9571-480e-bd3b-6ba8cfc4228c None None] MFA Rules not processed for user `fd3eb9e77dc64869b03b1190cf30e54b`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:35.865667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d23e4ae0-9571-480e-bd3b-6ba8cfc4228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:35.873147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 727/1474] 10.222.0.32 () {58 vars in 1059 bytes} [Wed May 4 14:46:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:35.881029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d6533-b07b-4748-b95d-54eb15d49053 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:35.893404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d6533-b07b-4748-b95d-54eb15d49053 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:35.897192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25539531-472e-48ec-a355-4a0b30436f60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:35.897854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25539531-472e-48ec-a355-4a0b30436f60 service glance] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:35.898155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25539531-472e-48ec-a355-4a0b30436f60 service glance] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:35.898470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25539531-472e-48ec-a355-4a0b30436f60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:35.908482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25539531-472e-48ec-a355-4a0b30436f60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:35.909894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25539531-472e-48ec-a355-4a0b30436f60 service glance] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:35.919691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 748/1475] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:46:38.257976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c25d9b-9da5-46bc-ab71-968d21552f72 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:38.258586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c25d9b-9da5-46bc-ab71-968d21552f72 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:38.258586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c25d9b-9da5-46bc-ab71-968d21552f72 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:38.259917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 728/1476] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:38.265235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8e04d-1729-4389-8b76-1aa3214924ad None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:38.265697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8e04d-1729-4389-8b76-1aa3214924ad None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:38.265803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8e04d-1729-4389-8b76-1aa3214924ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:38.285110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-10e8e04d-1729-4389-8b76-1aa3214924ad None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:38.286338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-10e8e04d-1729-4389-8b76-1aa3214924ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:38.295492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 749/1477] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:38.318796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c25d9b-9da5-46bc-ab71-968d21552f72 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:38.333315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c25d9b-9da5-46bc-ab71-968d21552f72 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:38.339228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6ca78-e0ca-4cd8-92b0-0d57b674cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:38.339896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6ca78-e0ca-4cd8-92b0-0d57b674cbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:38.340178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6ca78-e0ca-4cd8-92b0-0d57b674cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:38.340448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6ca78-e0ca-4cd8-92b0-0d57b674cbd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:38.342244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6ca78-e0ca-4cd8-92b0-0d57b674cbd1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:38.343020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-caf6ca78-e0ca-4cd8-92b0-0d57b674cbd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:46:38.343020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:46:38.343020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:46:38.344168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6ca78-e0ca-4cd8-92b0-0d57b674cbd1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:38.363752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 729/1478] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 4 14:46:38 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:46:40.622827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc25f7-7a0c-418c-8518-6ab36169023d None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:40.623403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc25f7-7a0c-418c-8518-6ab36169023d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:40.623403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc25f7-7a0c-418c-8518-6ab36169023d None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:40.624679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 750/1479] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:40.630453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac79bd-4f98-4725-a5d4-2621130866d5 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:40.630921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac79bd-4f98-4725-a5d4-2621130866d5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:40.631024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac79bd-4f98-4725-a5d4-2621130866d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:40.651124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-49ac79bd-4f98-4725-a5d4-2621130866d5 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:40.652365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-49ac79bd-4f98-4725-a5d4-2621130866d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:40.661820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 730/1480] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:40.703756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc25f7-7a0c-418c-8518-6ab36169023d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:40.719444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc25f7-7a0c-418c-8518-6ab36169023d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:40.726615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171b37c-a703-4717-b0cd-b33c797ffc79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:40.727278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171b37c-a703-4717-b0cd-b33c797ffc79 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:40.727565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171b37c-a703-4717-b0cd-b33c797ffc79 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:40.727836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171b37c-a703-4717-b0cd-b33c797ffc79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:40.737015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171b37c-a703-4717-b0cd-b33c797ffc79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:40.738605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171b37c-a703-4717-b0cd-b33c797ffc79 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:40.748206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 751/1481] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:46:43.174326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb035d11-1630-41e4-b4da-921965510e00 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:43.174968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb035d11-1630-41e4-b4da-921965510e00 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:43.174968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb035d11-1630-41e4-b4da-921965510e00 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:43.176262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 731/1482] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:43.181739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e8895-f795-4a09-a2ec-6450ecd7bc40 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:43.182203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e8895-f795-4a09-a2ec-6450ecd7bc40 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:43.182422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e8895-f795-4a09-a2ec-6450ecd7bc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:43.201010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-600e8895-f795-4a09-a2ec-6450ecd7bc40 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:43.202704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-600e8895-f795-4a09-a2ec-6450ecd7bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:43.211878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 752/1483] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:43.250741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb035d11-1630-41e4-b4da-921965510e00 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:43.262937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb035d11-1630-41e4-b4da-921965510e00 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:43.267202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05f599-7ba8-457a-8734-aedb07f9e5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:43.267841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f599-7ba8-457a-8734-aedb07f9e5dc service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:43.268122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f599-7ba8-457a-8734-aedb07f9e5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:43.268395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f599-7ba8-457a-8734-aedb07f9e5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:43.278111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05f599-7ba8-457a-8734-aedb07f9e5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:43.279535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05f599-7ba8-457a-8734-aedb07f9e5dc service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:43.289037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 732/1484] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:46:45.594432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70c2c-cd6e-445b-a9dd-e6b0259e4301 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:45.595026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70c2c-cd6e-445b-a9dd-e6b0259e4301 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:45.595026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b70c2c-cd6e-445b-a9dd-e6b0259e4301 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:45.596368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 753/1485] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:45.602221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e320b1-6913-45cf-8760-c74d794e5179 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:45.602791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e320b1-6913-45cf-8760-c74d794e5179 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:45.602791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e320b1-6913-45cf-8760-c74d794e5179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:45.621809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-06e320b1-6913-45cf-8760-c74d794e5179 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:45.623006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-06e320b1-6913-45cf-8760-c74d794e5179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:45.632609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 733/1486] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:45.672286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70c2c-cd6e-445b-a9dd-e6b0259e4301 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:45.684654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b70c2c-cd6e-445b-a9dd-e6b0259e4301 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:45.689025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4923f8-8bea-460b-a172-506f579fbf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:45.689672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4923f8-8bea-460b-a172-506f579fbf53 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:45.689957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4923f8-8bea-460b-a172-506f579fbf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:45.690236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4923f8-8bea-460b-a172-506f579fbf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:45.699876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4923f8-8bea-460b-a172-506f579fbf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:45.701359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4923f8-8bea-460b-a172-506f579fbf53 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:45.714079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 754/1487] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:46:48.037116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c86f0-34b7-4d4b-8d62-ed40ec3c70a9 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:48.037708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c86f0-34b7-4d4b-8d62-ed40ec3c70a9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:48.037708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c86f0-34b7-4d4b-8d62-ed40ec3c70a9 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:48.039092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 734/1488] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:48.044593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90e3e9-b197-45da-aabc-bfaa2ecfbfd9 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:48.045119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90e3e9-b197-45da-aabc-bfaa2ecfbfd9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:48.045234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90e3e9-b197-45da-aabc-bfaa2ecfbfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:48.063503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-dc90e3e9-b197-45da-aabc-bfaa2ecfbfd9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:48.065793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-dc90e3e9-b197-45da-aabc-bfaa2ecfbfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:48.074758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 755/1489] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:48.114260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c86f0-34b7-4d4b-8d62-ed40ec3c70a9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:48.126516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c86f0-34b7-4d4b-8d62-ed40ec3c70a9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:48.130396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e56631-2cbb-4b43-99f0-60b65fa1c8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:48.131037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56631-2cbb-4b43-99f0-60b65fa1c8de service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:48.131317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56631-2cbb-4b43-99f0-60b65fa1c8de service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:48.131589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56631-2cbb-4b43-99f0-60b65fa1c8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:48.141847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e56631-2cbb-4b43-99f0-60b65fa1c8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:48.143286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e56631-2cbb-4b43-99f0-60b65fa1c8de service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:48.153366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 735/1490] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:46:50.591988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a697cbb-60f5-4984-8a79-d68c0b13d8a1 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:50.592518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a697cbb-60f5-4984-8a79-d68c0b13d8a1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:50.592518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a697cbb-60f5-4984-8a79-d68c0b13d8a1 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:50.593868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 756/1491] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 4 14:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:46:50.599653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6c859-fe97-4030-842c-65c3939ac8fe None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:50.600164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6c859-fe97-4030-842c-65c3939ac8fe None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:50.600265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6c859-fe97-4030-842c-65c3939ac8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:50.619030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-15b6c859-fe97-4030-842c-65c3939ac8fe None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:46:50.620306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-15b6c859-fe97-4030-842c-65c3939ac8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:46:50.629685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 736/1492] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 4 14:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:46:50.667817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a697cbb-60f5-4984-8a79-d68c0b13d8a1 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:46:50.679739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a697cbb-60f5-4984-8a79-d68c0b13d8a1 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:46:50.684124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e0d66-a28e-4f36-a440-50b0dc2b4484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:46:50.684770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e0d66-a28e-4f36-a440-50b0dc2b4484 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:46:50.685088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e0d66-a28e-4f36-a440-50b0dc2b4484 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:46:50.685362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e0d66-a28e-4f36-a440-50b0dc2b4484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:46:50.695573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e0d66-a28e-4f36-a440-50b0dc2b4484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:46:50.697037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e0d66-a28e-4f36-a440-50b0dc2b4484 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:46:50.707311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 757/1493] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:48:18.376606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0521c05-ee3f-4c58-93d4-34f59215c8da None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.377340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0521c05-ee3f-4c58-93d4-34f59215c8da None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.377340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0521c05-ee3f-4c58-93d4-34f59215c8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.396599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-f0521c05-ee3f-4c58-93d4-34f59215c8da None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:48:18.397859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f0521c05-ee3f-4c58-93d4-34f59215c8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:48:18.407339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 737/1494] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:48:18.415404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d706-4cf9-4f97-a73f-0128ae80f1f0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.415939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d706-4cf9-4f97-a73f-0128ae80f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.415939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30d706-4cf9-4f97-a73f-0128ae80f1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.434400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2b30d706-4cf9-4f97-a73f-0128ae80f1f0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:48:18.435629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2b30d706-4cf9-4f97-a73f-0128ae80f1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:48:18.444198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 758/1495] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:48:18.448935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0521c05-ee3f-4c58-93d4-34f59215c8da None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:18.458514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0521c05-ee3f-4c58-93d4-34f59215c8da None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:18.463529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:18.464254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.464532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.464822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.465562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:48:18.465868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:18.466888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:18.466888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:48:18.466888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:48:18.467773 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:18.479797 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 738/1496] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:48:18.488321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30d706-4cf9-4f97-a73f-0128ae80f1f0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:18.497553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30d706-4cf9-4f97-a73f-0128ae80f1f0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:18.502314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:18.502981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.503262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.503534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.504240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:18.506075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING py.warnings [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:18.506075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: warnings.warn(msg) May 04 14:48:18.506075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: May 04 14:48:18.507270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:18.548483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 759/1497] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:48:18 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:48:18.552719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:18.562794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce87ef-790d-45d7-82a7-c0716bbe5f84 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:18.621405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:18.621997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.622297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.622568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.623258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:18.624317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:18.624317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:48:18.624317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:48:18.625543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:18.649635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 739/1498] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:48:18 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:48:18.654016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:18.664956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a0d09-6653-44cb-9c7c-342bbf73bba0 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:18.669847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:18.670502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.670792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.671094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.671842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:18.672907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING py.warnings [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:18.672907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: warnings.warn(msg) May 04 14:48:18.672907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: May 04 14:48:18.673819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:18.689323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 760/1499] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:48:18.693909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:18.705282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c6397-64b7-4b91-afc0-9ac832ce034b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:18.710464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005662-1e3a-431c-ab28-d4b0b7511010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:18.711129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005662-1e3a-431c-ab28-d4b0b7511010 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.711415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005662-1e3a-431c-ab28-d4b0b7511010 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.711663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005662-1e3a-431c-ab28-d4b0b7511010 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.712426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005662-1e3a-431c-ab28-d4b0b7511010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:18.713478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-08005662-1e3a-431c-ab28-d4b0b7511010 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:18.713478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:48:18.713478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:48:18.714362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005662-1e3a-431c-ab28-d4b0b7511010 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:18.729558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 740/1500] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:48:18.734398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:18.745019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dacad7-1c94-4959-a365-7f43f10f6ac9 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:18.749842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:18.750768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.751046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.751313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] PATH_INFO: `/v3/projects/998700b8e84d4fcd8e73ed0b0c00e4d6/users/96b90b26b6644b7d9561329bfa630599/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.766780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=998700b8e84d4fcd8e73ed0b0c00e4d6, user_id=96b90b26b6644b7d9561329bfa630599, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:18.768246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING py.warnings [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:18.768246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: warnings.warn(msg) May 04 14:48:18.768246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: May 04 14:48:18.770218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:18.785174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 761/1501] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:48:18 2022] PUT /identity/v3/projects/998700b8e84d4fcd8e73ed0b0c00e4d6/users/96b90b26b6644b7d9561329bfa630599/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:48:18.797113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dde7-7fe6-41b6-ac8f-3f147816702e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.797544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dde7-7fe6-41b6-ac8f-3f147816702e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.797801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794dde7-7fe6-41b6-ac8f-3f147816702e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.824775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6794dde7-7fe6-41b6-ac8f-3f147816702e None None] MFA Rules not processed for user `96b90b26b6644b7d9561329bfa630599`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:48:18.826042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6794dde7-7fe6-41b6-ac8f-3f147816702e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:48:18.891783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 741/1502] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:48:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:48:18.903370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:18.975978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a329c-9f25-4e18-b0ca-bca5dec7c3a3 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:18.983175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:18.983829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:18.984101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:18.984364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:18.993403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:18.995034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:19.004204 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 762/1503] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:48:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:48:19.077697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aee2aa-eaea-49b9-9c37-c5cc327f9b1b None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:19.077981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aee2aa-eaea-49b9-9c37-c5cc327f9b1b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:19.078256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aee2aa-eaea-49b9-9c37-c5cc327f9b1b None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:19.079527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 742/1504] 10.222.0.32 () {52 vars in 829 bytes} [Wed May 4 14:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:48:19.083465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:19.148688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84bc07-f708-49b2-8dd2-b1184dc62423 service nova] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:19.153738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:19.154628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:19.154908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:19.155181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] PATH_INFO: `/v3/users/96b90b26b6644b7d9561329bfa630599` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:19.156410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b90b26b6644b7d9561329bfa630599)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:19.157485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING py.warnings [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:19.157485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: warnings.warn(msg) May 04 14:48:19.157485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: May 04 14:48:19.158689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:19.222467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b90b26b6644b7d9561329bfa630599'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:19.240615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b90b26b6644b7d9561329bfa630599'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:19.241401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b90b26b6644b7d9561329bfa630599'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:19.242168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b90b26b6644b7d9561329bfa630599'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:19.251475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b90b26b6644b7d9561329bfa630599'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:19.270480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 763/1505] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:48:19 2022] DELETE /identity/v3/users/96b90b26b6644b7d9561329bfa630599 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:48:19.275451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:19.275875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:19.276153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:19.298126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:48:19.299938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:48:19.378375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 743/1506] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:48:19.389076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:19.390572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:48:19.454133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab10f8-307a-4f6f-9c49-8351d8453d7f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:19.458437 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d0722-94cc-46f9-8b0f-b35c359c9e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:19.459114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0722-94cc-46f9-8b0f-b35c359c9e87 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:19.459393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0722-94cc-46f9-8b0f-b35c359c9e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:19.459649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d0722-94cc-46f9-8b0f-b35c359c9e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:19.477178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d0722-94cc-46f9-8b0f-b35c359c9e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:19.478896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d0722-94cc-46f9-8b0f-b35c359c9e87 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:19.488784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 764/1507] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:48:19.874100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:48:19.875573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:48:19.887150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930760-6285-4953-92cc-ab028ca75fcf None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:48:19.891948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:48:19.893221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:48:19.893897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:48:19.894248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] PATH_INFO: `/v3/projects/998700b8e84d4fcd8e73ed0b0c00e4d6` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:48:19.895439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] RBAC: Authorizing `identity:delete_project(project_id=998700b8e84d4fcd8e73ed0b0c00e4d6)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:48:19.896460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:48:19.896460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:48:19.896460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:48:19.897656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:48:19.969219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998700b8e84d4fcd8e73ed0b0c00e4d6', 'request_id': 'req-970e2529-26ff-4c71-9130-88d66bd6d5db'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:19.982814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998700b8e84d4fcd8e73ed0b0c00e4d6', 'request_id': 'req-970e2529-26ff-4c71-9130-88d66bd6d5db'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:19.989323 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998700b8e84d4fcd8e73ed0b0c00e4d6', 'request_id': 'req-970e2529-26ff-4c71-9130-88d66bd6d5db'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:20.005796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] The token cache is being invalidate because project 998700b8e84d4fcd8e73ed0b0c00e4d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:48:20.006078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:20.006688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-970e2529-26ff-4c71-9130-88d66bd6d5db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:48:20.008364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 744/1508] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:48:19 2022] DELETE /identity/v3/projects/998700b8e84d4fcd8e73ed0b0c00e4d6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:50:06.971435 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009cce-4e64-4d5b-8cc8-5c29f2fb9037 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:06.972070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009cce-4e64-4d5b-8cc8-5c29f2fb9037 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:06.972070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009cce-4e64-4d5b-8cc8-5c29f2fb9037 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:06.973437 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 765/1509] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:06.978648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bb481-7fda-45bc-b19a-15a4fd0465d3 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:06.979133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bb481-7fda-45bc-b19a-15a4fd0465d3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:06.979133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bb481-7fda-45bc-b19a-15a4fd0465d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:06.999785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-f34bb481-7fda-45bc-b19a-15a4fd0465d3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:07.001094 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f34bb481-7fda-45bc-b19a-15a4fd0465d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:07.072914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 745/1510] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:07.231142 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009cce-4e64-4d5b-8cc8-5c29f2fb9037 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:07.233274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-80009cce-4e64-4d5b-8cc8-5c29f2fb9037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:07.316806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009cce-4e64-4d5b-8cc8-5c29f2fb9037 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:07.323386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebd673-2d44-48a7-b8f6-3ace858a36f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:07.323969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebd673-2d44-48a7-b8f6-3ace858a36f0 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:07.324261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebd673-2d44-48a7-b8f6-3ace858a36f0 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:07.324534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebd673-2d44-48a7-b8f6-3ace858a36f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:07.334030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebd673-2d44-48a7-b8f6-3ace858a36f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:07.335655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebd673-2d44-48a7-b8f6-3ace858a36f0 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:07.345571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 766/1511] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:50:07.574553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f827b-65cb-4b7a-ae11-863195260d58 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:07.574785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f827b-65cb-4b7a-ae11-863195260d58 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:07.575104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f827b-65cb-4b7a-ae11-863195260d58 None None] PATH_INFO: `/v3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:07.576524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 746/1512] 10.222.0.49 () {54 vars in 925 bytes} [Wed May 4 14:50:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:50:07.583450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fff28-29a9-4e0c-8ed4-08729cc80634 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:07.583724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fff28-29a9-4e0c-8ed4-08729cc80634 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:07.584030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fff28-29a9-4e0c-8ed4-08729cc80634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:07.602739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4a0fff28-29a9-4e0c-8ed4-08729cc80634 None None] MFA Rules not processed for user `7ac21bca8da642338571f821ef5ea121`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:07.603989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4a0fff28-29a9-4e0c-8ed4-08729cc80634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:07.689764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 767/1513] 10.222.0.49 () {58 vars in 1015 bytes} [Wed May 4 14:50:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:08.569895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6b55-27af-41ad-ac68-05fb1beaecb9 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:08.570494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6b55-27af-41ad-ac68-05fb1beaecb9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:08.570494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6b55-27af-41ad-ac68-05fb1beaecb9 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:08.571845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:08.577084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92f0a-79cb-4a76-8114-70e6b9875c43 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:08.577552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92f0a-79cb-4a76-8114-70e6b9875c43 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:08.577552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92f0a-79cb-4a76-8114-70e6b9875c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:08.595901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-64b92f0a-79cb-4a76-8114-70e6b9875c43 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:08.597151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-64b92f0a-79cb-4a76-8114-70e6b9875c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:08.606105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:08.757078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6b55-27af-41ad-ac68-05fb1beaecb9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:08.774642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6b55-27af-41ad-ac68-05fb1beaecb9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:08.780859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6605917-3b02-4ad4-9a47-d1751f65ffec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:08.781506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6605917-3b02-4ad4-9a47-d1751f65ffec service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:08.781803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6605917-3b02-4ad4-9a47-d1751f65ffec service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:08.782059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6605917-3b02-4ad4-9a47-d1751f65ffec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:08.791344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6605917-3b02-4ad4-9a47-d1751f65ffec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:08.793023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6605917-3b02-4ad4-9a47-d1751f65ffec service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:08.803275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 748/1516] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:50:11.492664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa0bd-1e91-4c40-86a1-e381b47c091b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:11.493258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa0bd-1e91-4c40-86a1-e381b47c091b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:11.493258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa0bd-1e91-4c40-86a1-e381b47c091b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:11.494637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 769/1517] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 4 14:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:11.499872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46d4b-e137-46e6-b3b0-d4ea1028ea82 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:11.500391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46d4b-e137-46e6-b3b0-d4ea1028ea82 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:11.500391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46d4b-e137-46e6-b3b0-d4ea1028ea82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:11.519881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-68c46d4b-e137-46e6-b3b0-d4ea1028ea82 None None] MFA Rules not processed for user `7ac21bca8da642338571f821ef5ea121`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:11.521203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-68c46d4b-e137-46e6-b3b0-d4ea1028ea82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:11.530193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 749/1518] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 4 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:11.537960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caa0bd-1e91-4c40-86a1-e381b47c091b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:11.552506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caa0bd-1e91-4c40-86a1-e381b47c091b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:11.557822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ce3ac-c0fa-4934-b15f-b40c9438283c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac21bca8da642338571f821ef5ea121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:11.558455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce3ac-c0fa-4934-b15f-b40c9438283c service placement] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:11.558733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce3ac-c0fa-4934-b15f-b40c9438283c service placement] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:11.559010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ce3ac-c0fa-4934-b15f-b40c9438283c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:11.567668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ce3ac-c0fa-4934-b15f-b40c9438283c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:11.569369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ce3ac-c0fa-4934-b15f-b40c9438283c service placement] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:11.581062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 770/1519] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:50:20.591535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93a1f-6105-4f7a-84ce-5d44ebef3a85 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:20.592121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93a1f-6105-4f7a-84ce-5d44ebef3a85 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:20.592121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a93a1f-6105-4f7a-84ce-5d44ebef3a85 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:20.593434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 750/1520] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:20.598748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca0b9a-3230-44dd-a0ea-a8a8d4d453eb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:20.599247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca0b9a-3230-44dd-a0ea-a8a8d4d453eb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:20.599350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca0b9a-3230-44dd-a0ea-a8a8d4d453eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:20.618546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-56ca0b9a-3230-44dd-a0ea-a8a8d4d453eb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:20.619776 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-56ca0b9a-3230-44dd-a0ea-a8a8d4d453eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:20.629108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 771/1521] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:20.778632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a93a1f-6105-4f7a-84ce-5d44ebef3a85 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:20.794315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a93a1f-6105-4f7a-84ce-5d44ebef3a85 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:20.800381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff48e41-2c43-49b0-a6d6-ffcaf1b1c1f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:20.801124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48e41-2c43-49b0-a6d6-ffcaf1b1c1f6 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:20.801416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48e41-2c43-49b0-a6d6-ffcaf1b1c1f6 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:20.801648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48e41-2c43-49b0-a6d6-ffcaf1b1c1f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:20.811079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff48e41-2c43-49b0-a6d6-ffcaf1b1c1f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:20.812703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff48e41-2c43-49b0-a6d6-ffcaf1b1c1f6 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:20.823306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 751/1522] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:50:22.078939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd39e-bddd-47f0-ac73-3b511858b705 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:22.079512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd39e-bddd-47f0-ac73-3b511858b705 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:22.079512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd39e-bddd-47f0-ac73-3b511858b705 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:22.080824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 772/1523] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 4 14:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:22.085928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052a56-77b1-40f7-ab88-d8377d73755c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:22.086406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052a56-77b1-40f7-ab88-d8377d73755c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:22.086511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052a56-77b1-40f7-ab88-d8377d73755c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:22.105256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-69052a56-77b1-40f7-ab88-d8377d73755c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:22.106465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-69052a56-77b1-40f7-ab88-d8377d73755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:22.115713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 752/1524] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 4 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:22.263512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd39e-bddd-47f0-ac73-3b511858b705 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:22.280124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd39e-bddd-47f0-ac73-3b511858b705 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:22.286188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7497203-3c99-477d-afa7-97ec7e0f434a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:22.286853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7497203-3c99-477d-afa7-97ec7e0f434a service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:22.287133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7497203-3c99-477d-afa7-97ec7e0f434a service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:22.287406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7497203-3c99-477d-afa7-97ec7e0f434a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:22.296057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7497203-3c99-477d-afa7-97ec7e0f434a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:22.297683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7497203-3c99-477d-afa7-97ec7e0f434a service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:22.310139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 773/1525] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:50:26.601969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b7456-e9d5-4f93-9ff1-b6a7b1304317 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:26.602952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b7456-e9d5-4f93-9ff1-b6a7b1304317 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:26.602952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b7456-e9d5-4f93-9ff1-b6a7b1304317 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:26.604574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 753/1526] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:26.610822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24ea57-f541-40cc-8b0c-f0f1efbbda18 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:26.611459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24ea57-f541-40cc-8b0c-f0f1efbbda18 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:26.611459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24ea57-f541-40cc-8b0c-f0f1efbbda18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:26.631244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ff24ea57-f541-40cc-8b0c-f0f1efbbda18 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:26.632719 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ff24ea57-f541-40cc-8b0c-f0f1efbbda18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:26.642447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 774/1527] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:26.681412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b7456-e9d5-4f93-9ff1-b6a7b1304317 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:26.683628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f83b7456-e9d5-4f93-9ff1-b6a7b1304317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:26.747563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b7456-e9d5-4f93-9ff1-b6a7b1304317 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:26.751724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb2f95d-867e-4b6f-8225-db63821f4b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:26.752381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2f95d-867e-4b6f-8225-db63821f4b48 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:26.752690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2f95d-867e-4b6f-8225-db63821f4b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:26.753001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb2f95d-867e-4b6f-8225-db63821f4b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:26.763837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2f95d-867e-4b6f-8225-db63821f4b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:26.765571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb2f95d-867e-4b6f-8225-db63821f4b48 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:26.777161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 754/1528] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:50:29.099356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23b8e-981a-48fc-994c-e14a86705b11 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:29.100031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23b8e-981a-48fc-994c-e14a86705b11 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:29.100031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f23b8e-981a-48fc-994c-e14a86705b11 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:29.101415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 775/1529] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:50:29.107414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899730b-4442-4455-ada6-daf1ff958bbd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:29.108296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899730b-4442-4455-ada6-daf1ff958bbd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:29.108296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899730b-4442-4455-ada6-daf1ff958bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:29.127575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e899730b-4442-4455-ada6-daf1ff958bbd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:29.128837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e899730b-4442-4455-ada6-daf1ff958bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:29.139910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 755/1530] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:29.178512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f23b8e-981a-48fc-994c-e14a86705b11 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:29.180423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d5f23b8e-981a-48fc-994c-e14a86705b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:29.194635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f23b8e-981a-48fc-994c-e14a86705b11 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:29.198569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8eb9e4-85cf-4af9-bd19-e49f3ca9f425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:29.199253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8eb9e4-85cf-4af9-bd19-e49f3ca9f425 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:29.199535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8eb9e4-85cf-4af9-bd19-e49f3ca9f425 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:29.199798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8eb9e4-85cf-4af9-bd19-e49f3ca9f425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:29.210623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8eb9e4-85cf-4af9-bd19-e49f3ca9f425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:29.212229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8eb9e4-85cf-4af9-bd19-e49f3ca9f425 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:29.223604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 776/1531] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:50:33.142313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680523d-76ae-45bc-b8cf-5860247f5c7f None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:33.143182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680523d-76ae-45bc-b8cf-5860247f5c7f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:33.143182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680523d-76ae-45bc-b8cf-5860247f5c7f None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:33.144766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 756/1532] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:33.151480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da135d2d-b69e-42e8-b11a-ac23548da05b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:33.152109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da135d2d-b69e-42e8-b11a-ac23548da05b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:33.152109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da135d2d-b69e-42e8-b11a-ac23548da05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:33.172611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-da135d2d-b69e-42e8-b11a-ac23548da05b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:33.173892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-da135d2d-b69e-42e8-b11a-ac23548da05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:33.182849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 777/1533] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:33.484313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680523d-76ae-45bc-b8cf-5860247f5c7f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:33.486216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3680523d-76ae-45bc-b8cf-5860247f5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:33.581536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680523d-76ae-45bc-b8cf-5860247f5c7f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:33.588279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02d527-c131-4f6e-9111-d653b540b6ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:33.589127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d527-c131-4f6e-9111-d653b540b6ba service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:33.589405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d527-c131-4f6e-9111-d653b540b6ba service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:33.589673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d527-c131-4f6e-9111-d653b540b6ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:33.601824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02d527-c131-4f6e-9111-d653b540b6ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:33.603399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02d527-c131-4f6e-9111-d653b540b6ba service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:33.615710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 757/1534] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:50:36.124846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec30b51-3bc0-485c-a9e1-12d8b76def7b None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:36.125765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec30b51-3bc0-485c-a9e1-12d8b76def7b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:36.125765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec30b51-3bc0-485c-a9e1-12d8b76def7b None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:36.127491 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 778/1535] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 4 14:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:36.133611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47ae19-949f-4771-8f2d-76b4924080ae None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:36.134262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47ae19-949f-4771-8f2d-76b4924080ae None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:36.134262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47ae19-949f-4771-8f2d-76b4924080ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:36.154684 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-de47ae19-949f-4771-8f2d-76b4924080ae None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:36.155917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-de47ae19-949f-4771-8f2d-76b4924080ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:36.164491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 758/1536] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 4 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:36.447912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6967ad-63c7-4d2f-97cb-6bf61a2b0d1c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:36.448397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6967ad-63c7-4d2f-97cb-6bf61a2b0d1c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:36.448935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6967ad-63c7-4d2f-97cb-6bf61a2b0d1c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:36.451595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 779/1537] 10.222.0.32 () {54 vars in 963 bytes} [Wed May 4 14:50:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:50:36.457636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272174b-bd23-4d30-a22f-66bc7ea3c585 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:36.458049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272174b-bd23-4d30-a22f-66bc7ea3c585 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:36.458428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272174b-bd23-4d30-a22f-66bc7ea3c585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:36.482648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b272174b-bd23-4d30-a22f-66bc7ea3c585 None None] MFA Rules not processed for user `a4aed1720aea4ffa975f7936a1bdb3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:50:36.483887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b272174b-bd23-4d30-a22f-66bc7ea3c585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:50:36.491230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 759/1538] 10.222.0.32 () {58 vars in 1060 bytes} [Wed May 4 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:50:36.500319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6967ad-63c7-4d2f-97cb-6bf61a2b0d1c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:50:36.513959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6967ad-63c7-4d2f-97cb-6bf61a2b0d1c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:50:36.518898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5db8f2-ed1b-4057-92c8-405e8f585a67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:50:36.519726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5db8f2-ed1b-4057-92c8-405e8f585a67 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:50:36.520067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5db8f2-ed1b-4057-92c8-405e8f585a67 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:50:36.520427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5db8f2-ed1b-4057-92c8-405e8f585a67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:50:36.531674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5db8f2-ed1b-4057-92c8-405e8f585a67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:50:36.533383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5db8f2-ed1b-4057-92c8-405e8f585a67 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:50:36.544959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 780/1539] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:09.907452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95375b68-772d-4810-9d37-7aa4ae0c0410 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:09.908140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95375b68-772d-4810-9d37-7aa4ae0c0410 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:09.908468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95375b68-772d-4810-9d37-7aa4ae0c0410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:09.939408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-95375b68-772d-4810-9d37-7aa4ae0c0410 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:09.944740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-95375b68-772d-4810-9d37-7aa4ae0c0410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:09.954930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 760/1540] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:09.966238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75834ca7-8c84-40da-87d2-a2271478537f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:09.966865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75834ca7-8c84-40da-87d2-a2271478537f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:09.966865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75834ca7-8c84-40da-87d2-a2271478537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:09.997436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-75834ca7-8c84-40da-87d2-a2271478537f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:09.998700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-75834ca7-8c84-40da-87d2-a2271478537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.019998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 781/1541] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:10.025107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95375b68-772d-4810-9d37-7aa4ae0c0410 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.043289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95375b68-772d-4810-9d37-7aa4ae0c0410 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.059775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.061808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.062939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.063879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.065312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:10.066186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.070010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.094679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 761/1542] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:10.110067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75834ca7-8c84-40da-87d2-a2271478537f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.135826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75834ca7-8c84-40da-87d2-a2271478537f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.144662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.145377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.145657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.145931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.146644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.148766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.195396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 782/1543] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:10.199851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.214246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d1179-9e90-40a1-b562-4148caf833fc admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.293865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.295213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.295721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.296354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.297370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.299653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.335026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 762/1544] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:10.340081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.360139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22018d19-cf48-4b26-b3b8-032232d50a99 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.374000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9bad9-5074-498e-a5cf-f0ad8d8c9fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.374922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9bad9-5074-498e-a5cf-f0ad8d8c9fcb admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.375262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9bad9-5074-498e-a5cf-f0ad8d8c9fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.375747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9bad9-5074-498e-a5cf-f0ad8d8c9fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.376845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9bad9-5074-498e-a5cf-f0ad8d8c9fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.380357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9bad9-5074-498e-a5cf-f0ad8d8c9fcb admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.411851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 783/1545] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:10.413349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.415379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d429-f708-45e4-ae12-fd59987ca8ec None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.415923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d429-f708-45e4-ae12-fd59987ca8ec None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.416393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d429-f708-45e4-ae12-fd59987ca8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.435911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5dbe7-facb-435c-8c03-3ff35adc3e81 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.447632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c20ee-28cc-4162-8b5c-d71405f6896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.449159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c20ee-28cc-4162-8b5c-d71405f6896c admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.449683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c20ee-28cc-4162-8b5c-d71405f6896c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.450193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c20ee-28cc-4162-8b5c-d71405f6896c admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.451355 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c20ee-28cc-4162-8b5c-d71405f6896c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.453237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c20ee-28cc-4162-8b5c-d71405f6896c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.474909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d227d429-f708-45e4-ae12-fd59987ca8ec None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:10.475607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 763/1546] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:52:10.476557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d227d429-f708-45e4-ae12-fd59987ca8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.477563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb592397-9633-4c64-8370-16cfee0573bc None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.478096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb592397-9633-4c64-8370-16cfee0573bc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.478598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb592397-9633-4c64-8370-16cfee0573bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.495106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 784/1547] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:10.496087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227d429-f708-45e4-ae12-fd59987ca8ec None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.517030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227d429-f708-45e4-ae12-fd59987ca8ec None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.525987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-fb592397-9633-4c64-8370-16cfee0573bc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:10.528227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fb592397-9633-4c64-8370-16cfee0573bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.530603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.531450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.531990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.532456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] PATH_INFO: `/v3/projects/9888acb2dc5d4b95881080cf62d40844/users/366e2408410f4c2c930881f42c2b18c4/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.558765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 764/1548] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:10.560670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c349c2-3a93-4af0-93e2-16dce4779196 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.561273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c349c2-3a93-4af0-93e2-16dce4779196 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.562349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c349c2-3a93-4af0-93e2-16dce4779196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.581200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9888acb2dc5d4b95881080cf62d40844, user_id=366e2408410f4c2c930881f42c2b18c4, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.588074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.603789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a7c349c2-3a93-4af0-93e2-16dce4779196 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:10.605505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a7c349c2-3a93-4af0-93e2-16dce4779196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.631606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 765/1549] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:10.633639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 785/1550] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:10 2022] PUT /identity/v3/projects/9888acb2dc5d4b95881080cf62d40844/users/366e2408410f4c2c930881f42c2b18c4/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:10.634251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412ae5-0a79-4de0-afb2-c7489c800348 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.634795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412ae5-0a79-4de0-afb2-c7489c800348 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.635314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412ae5-0a79-4de0-afb2-c7489c800348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.642387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.681730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-28412ae5-0a79-4de0-afb2-c7489c800348 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:10.683345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-28412ae5-0a79-4de0-afb2-c7489c800348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.774580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6f8cb-906f-4011-b28b-47b4a89330da admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.789340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.790802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.791097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.791380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.792166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:10.792479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.793615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING py.warnings [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 04 14:52:10.793615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: warnings.warn(msg) May 04 14:52:10.793615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: May 04 14:52:10.794542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b12974-9f15-4ecb-8a2d-0e30382c85b7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.800521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 766/1551] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:10.802232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb839c-cd58-4c30-9f8f-7992c5987214 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.802525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb839c-cd58-4c30-9f8f-7992c5987214 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.802842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb839c-cd58-4c30-9f8f-7992c5987214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.833176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 786/1552] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:10.835163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f4821-d6e0-41ff-a35b-fdd08839f6ec None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.835471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f4821-d6e0-41ff-a35b-fdd08839f6ec None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.835786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f4821-d6e0-41ff-a35b-fdd08839f6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.850370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-98cb839c-cd58-4c30-9f8f-7992c5987214 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:10.851779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-98cb839c-cd58-4c30-9f8f-7992c5987214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.879830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 767/1553] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:10.881541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b85da2-7e08-4f84-a097-a57a6cebd1d8 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.881891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b85da2-7e08-4f84-a097-a57a6cebd1d8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.882117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b85da2-7e08-4f84-a097-a57a6cebd1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.886707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4d3f4821-d6e0-41ff-a35b-fdd08839f6ec None None] MFA Rules not processed for user `366e2408410f4c2c930881f42c2b18c4`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:10.888002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4d3f4821-d6e0-41ff-a35b-fdd08839f6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.920188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-90b85da2-7e08-4f84-a097-a57a6cebd1d8 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:10.921510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-90b85da2-7e08-4f84-a097-a57a6cebd1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:10.932010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 768/1554] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:10.933250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b85da2-7e08-4f84-a097-a57a6cebd1d8 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.948312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b85da2-7e08-4f84-a097-a57a6cebd1d8 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.953860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.954545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.954841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.955120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.955875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:10.956189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:10.957862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:10.970971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 769/1555] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:10.971832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:10.989917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e009a0-a6c4-41e3-818e-77e5ebe5148f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:10.991758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 787/1556] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:10.993244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce94dcb-df66-465e-ab4c-d86657d01981 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:10.993546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce94dcb-df66-465e-ab4c-d86657d01981 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:10.993826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce94dcb-df66-465e-ab4c-d86657d01981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:10.999110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b46d44-0358-444d-a607-2477969541f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:10.999848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46d44-0358-444d-a607-2477969541f7 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.000282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46d44-0358-444d-a607-2477969541f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.000636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46d44-0358-444d-a607-2477969541f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.001473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46d44-0358-444d-a607-2477969541f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.002562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-b6b46d44-0358-444d-a607-2477969541f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:52:11.002562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:52:11.002562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:52:11.003823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46d44-0358-444d-a607-2477969541f7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.017257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-dce94dcb-df66-465e-ab4c-d86657d01981 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.018971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-dce94dcb-df66-465e-ab4c-d86657d01981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.029148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 788/1557] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.030714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e1bdb-6c79-493e-993b-10bcdc727275 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.031210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e1bdb-6c79-493e-993b-10bcdc727275 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.031533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e1bdb-6c79-493e-993b-10bcdc727275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.042096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 770/1558] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:10 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.045366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c4f80-9a6c-4ae2-9aa1-60329e5eea27 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.045366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c4f80-9a6c-4ae2-9aa1-60329e5eea27 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.045366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c4f80-9a6c-4ae2-9aa1-60329e5eea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.055753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b57e1bdb-6c79-493e-993b-10bcdc727275 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.057038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b57e1bdb-6c79-493e-993b-10bcdc727275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.063288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-325c4f80-9a6c-4ae2-9aa1-60329e5eea27 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.064571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-325c4f80-9a6c-4ae2-9aa1-60329e5eea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.129208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 789/1559] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.130082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e1bdb-6c79-493e-993b-10bcdc727275 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.134287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 771/1560] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.134287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c4f80-9a6c-4ae2-9aa1-60329e5eea27 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.140655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e1bdb-6c79-493e-993b-10bcdc727275 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.143933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c4f80-9a6c-4ae2-9aa1-60329e5eea27 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.145764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.146430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.146713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.146989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.147685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.148805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0674a7-7163-428b-8fbe-ef2b1a22961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.149459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0674a7-7163-428b-8fbe-ef2b1a22961e admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.149637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.149780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0674a7-7163-428b-8fbe-ef2b1a22961e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.150045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0674a7-7163-428b-8fbe-ef2b1a22961e admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.150720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0674a7-7163-428b-8fbe-ef2b1a22961e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.152845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0674a7-7163-428b-8fbe-ef2b1a22961e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.180170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 790/1561] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.180170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.184827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 772/1562] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.185472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7987d-94db-4a59-a6f1-9f55776527cd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.186009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7987d-94db-4a59-a6f1-9f55776527cd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.186516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7987d-94db-4a59-a6f1-9f55776527cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.207287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0ef7987d-94db-4a59-a6f1-9f55776527cd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.208517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0ef7987d-94db-4a59-a6f1-9f55776527cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.262407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ec0bb-2dce-4560-a53e-d9d9e04c1bd3 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.274665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 773/1563] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.275390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7987d-94db-4a59-a6f1-9f55776527cd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.286841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7987d-94db-4a59-a6f1-9f55776527cd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.293263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91a49f-c0b0-44b2-9fba-46dbed785fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.293936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91a49f-c0b0-44b2-9fba-46dbed785fcb admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.294212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91a49f-c0b0-44b2-9fba-46dbed785fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.294476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91a49f-c0b0-44b2-9fba-46dbed785fcb admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.295172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91a49f-c0b0-44b2-9fba-46dbed785fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.297144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91a49f-c0b0-44b2-9fba-46dbed785fcb admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.328012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 774/1564] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/users => generated 287 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.328012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117a9f1-e0a9-44e0-b93f-f95261431cc0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.328012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117a9f1-e0a9-44e0-b93f-f95261431cc0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.328012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117a9f1-e0a9-44e0-b93f-f95261431cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.339631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.340685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.341113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.341484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.342650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:11.343170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.345036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.348706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-8117a9f1-e0a9-44e0-b93f-f95261431cc0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.349965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8117a9f1-e0a9-44e0-b93f-f95261431cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.357952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 791/1565] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:11.358915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.360951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 775/1566] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.362509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0816d-4ab0-4938-836a-a8b2f1245634 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.362983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0816d-4ab0-4938-836a-a8b2f1245634 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.363401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0816d-4ab0-4938-836a-a8b2f1245634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.370495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d1caa-8bd8-4a77-9096-442b439077ab admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.375452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.376222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.376575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.376954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.377798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:11.378109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.379786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.383045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-80d0816d-4ab0-4938-836a-a8b2f1245634 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.384235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-80d0816d-4ab0-4938-836a-a8b2f1245634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.392416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 792/1567] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:11.396073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 776/1568] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.396236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.396455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0816d-4ab0-4938-836a-a8b2f1245634 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.407542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0816d-4ab0-4938-836a-a8b2f1245634 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.407705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00aa7e-9ac4-47d2-a3d2-364858d75964 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.412093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.412224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e77c6-486e-45a3-bc22-b84b81dbf81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.412757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.412878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e77c6-486e-45a3-bc22-b84b81dbf81b admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.413067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.413163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e77c6-486e-45a3-bc22-b84b81dbf81b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.413342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.413445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e77c6-486e-45a3-bc22-b84b81dbf81b admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.414047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.414140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e77c6-486e-45a3-bc22-b84b81dbf81b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.415086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING py.warnings [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:52:11.415086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: warnings.warn(msg) May 04 14:52:11.415086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: May 04 14:52:11.416209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e77c6-486e-45a3-bc22-b84b81dbf81b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.416303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.437694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 777/1569] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.439111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4f3b-25ce-4157-b22e-8191dcb5ee30 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.439457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4f3b-25ce-4157-b22e-8191dcb5ee30 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.439686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd4f3b-25ce-4157-b22e-8191dcb5ee30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.441780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 793/1570] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.442791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.452320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd7118-0d18-4e03-8db5-e8a7c0c9657c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.457009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.457656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.457929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.458202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.458443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e8bd4f3b-25ce-4157-b22e-8191dcb5ee30 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.458937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.459703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e8bd4f3b-25ce-4157-b22e-8191dcb5ee30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.460572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.467762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 778/1571] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.469203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d81a9-5e5d-40d8-a3d7-eec7db33b0ae None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.469529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d81a9-5e5d-40d8-a3d7-eec7db33b0ae None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.469800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d81a9-5e5d-40d8-a3d7-eec7db33b0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.471881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 794/1572] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:11.472868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.482452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f56de4-e48b-40db-a2df-49a096b9708a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.487302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.487970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.488248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.488522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.489298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.489704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ba9d81a9-5e5d-40d8-a3d7-eec7db33b0ae None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:11.490967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ba9d81a9-5e5d-40d8-a3d7-eec7db33b0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:11.491242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.499590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 779/1573] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:11.500438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d81a9-5e5d-40d8-a3d7-eec7db33b0ae None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.509733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d81a9-5e5d-40d8-a3d7-eec7db33b0ae None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.515326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.516258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.516681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.517108 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.518105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:11.518422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.520035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.523550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 795/1574] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.524380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.536960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 780/1575] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:11.536960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.544425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77845ac6-aafe-4ff5-852c-047f46bcc6b2 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.596419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb449a6b-1140-4ea3-8459-a37fc7bade27 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.600765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.601469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.601633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.601781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.602129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.602280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.602397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.602673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.602853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.603413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.604594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.605370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.617840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 781/1576] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:11.617840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.629769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a04b-ae2f-4e0c-b086-1c34398caf79 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.635530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.636242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.636524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.636844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.637182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 796/1577] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.637553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.638010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.639184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.651823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 782/1578] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:11.652451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.662100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c44397-d56c-434f-a5fb-627058b714a0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.711281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70a99c-0725-4976-8bcb-c7c095bc9aa7 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.717483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.718079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.718357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.718630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.719536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:11.719847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.721618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.725367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.726271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.726648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.727015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.727911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.729890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.737090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 797/1579] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:11.738256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.752861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 783/1580] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:11.752861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.753221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34e584-44a2-4754-b900-081ee2a4693b admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.757227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.758049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.758331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.758607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.759356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:11.759668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.761338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.762574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dc56f-b381-4d12-99d7-3aeb512d3298 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.767760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.768440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.768715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.769022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.769725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.771686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.774570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 798/1581] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:11.775596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.793488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d9d13-f774-414b-a424-f0afe8c922d5 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.794520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 784/1582] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.795610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.799100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.799928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.800304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.800671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.801505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.803598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.805643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fbc471-6626-42c7-a73f-871817feb9c8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.810222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.811123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.811399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.811675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] PATH_INFO: `/v3/projects/df068037811443fbbea3944d108df9be/users/c7cf3d34bc3446adb439b637fe1b1a2b/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.833513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=df068037811443fbbea3944d108df9be, user_id=c7cf3d34bc3446adb439b637fe1b1a2b, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.834980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:52:11.834980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:52:11.834980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:52:11.838054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.843286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 799/1583] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:11.844237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.858597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 785/1584] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:11 2022] PUT /identity/v3/projects/df068037811443fbbea3944d108df9be/users/c7cf3d34bc3446adb439b637fe1b1a2b/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:11.860430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.920847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fbe63-4ace-4eaa-87df-ecedffd5f6bf admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.952255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e34091-c2a4-4fde-99b8-7f0534cf07ca admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.958850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.959771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.960328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.960774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.961713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.963518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.975867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 786/1585] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:11.976726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:11.988052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5e5e8-e99b-404b-a77e-3c149250a2f0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:11.993696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.994387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.994656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.994939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.995655 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:11.995886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:11.997038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:11.997435 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:11.997638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:11.997830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:11.998621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.000940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.032341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 800/1586] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:12.032341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.039174 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 787/1587] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:11 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:12.039174 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.044691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd870c-00cc-481a-aba9-96aa6994d908 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.110473 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.111386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.111662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.111934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] PATH_INFO: `/v3/projects/68496c30e63b44c3b89fb209c4160b0f/users/75c9f8cec68240e3a09dd4b8a95502bc/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.129599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68496c30e63b44c3b89fb209c4160b0f, user_id=75c9f8cec68240e3a09dd4b8a95502bc, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.130936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa2f47-40d4-41a0-8a0e-7a22668cd6f3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.134492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.138471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.139478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.139832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.140712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.142168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.145757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.155348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 801/1588] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:12 2022] PUT /identity/v3/projects/68496c30e63b44c3b89fb209c4160b0f/users/75c9f8cec68240e3a09dd4b8a95502bc/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:12.156282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.191198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 788/1589] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:12.192640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.224541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891263e-383f-4f24-b6db-281434acff06 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.282119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f1c60-f1a9-4e5d-abc5-4511543ec9fd admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.286984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.287679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.287973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.288254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.288995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.291011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.294452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.295126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.295401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.295668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.296397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.298146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.310460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 802/1590] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.311610 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.319264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 789/1591] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:12.319264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.322959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80899e33-2f84-4781-b6d0-9e54639bb71e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.328394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.329201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.329486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.329755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.330548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.332441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.334309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5bb7-c9fe-4592-a4ad-4bb0fab03811 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.339180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256c861-0229-41c0-976d-22077815a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.340087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256c861-0229-41c0-976d-22077815a00d admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.340515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256c861-0229-41c0-976d-22077815a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.340788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256c861-0229-41c0-976d-22077815a00d admin admin] PATH_INFO: `/v3/projects/958a8323b6914605b4767698e1510825/users/b4a70ea6369348bea63dc3eac7ac0380/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.343514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 803/1592] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.344446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.356835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d11e7b-683c-417a-b46b-120d547069c5 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.358813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256c861-0229-41c0-976d-22077815a00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=958a8323b6914605b4767698e1510825, user_id=b4a70ea6369348bea63dc3eac7ac0380, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.361736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256c861-0229-41c0-976d-22077815a00d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.361974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.362408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.362677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.362949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.363691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.365312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.377190 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 804/1593] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.381921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 790/1594] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:12 2022] PUT /identity/v3/projects/958a8323b6914605b4767698e1510825/users/b4a70ea6369348bea63dc3eac7ac0380/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:12.382125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.382237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b348e-27c6-46a6-ac1d-81797c31ca2e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.382237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b348e-27c6-46a6-ac1d-81797c31ca2e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.382237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b348e-27c6-46a6-ac1d-81797c31ca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.392213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ed37-6bf6-4b10-9946-9dac7992c7a6 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.410961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b57b348e-27c6-46a6-ac1d-81797c31ca2e None None] MFA Rules not processed for user `75c9f8cec68240e3a09dd4b8a95502bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:12.412364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b57b348e-27c6-46a6-ac1d-81797c31ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:12.451783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.452381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.452673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.452964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.453736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.455728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.477913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 805/1595] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:12.479118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.481262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 791/1596] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:12.482400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b348e-27c6-46a6-ac1d-81797c31ca2e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.492553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff90e1-26e7-4ec3-a8a1-a27f7641b71a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.493994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b348e-27c6-46a6-ac1d-81797c31ca2e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.499667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.500438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.500856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.501226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.502002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.502140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.503237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.503674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.503843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.504249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.505599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.509530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.515838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 792/1597] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.516774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.528464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8c7e1-a9f8-4663-a513-365dfb042f23 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.534874 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.535265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 806/1598] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/users => generated 288 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:12.535675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.537850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.537850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.538095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.538192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.539094 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.550971 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 793/1599] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.551938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.553369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb716e8-1d90-4455-9513-058f7226dd90 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.559377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b327f-177a-4854-b75f-5d7682d372e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.560495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b327f-177a-4854-b75f-5d7682d372e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.561003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b327f-177a-4854-b75f-5d7682d372e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.561291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b327f-177a-4854-b75f-5d7682d372e5 admin admin] PATH_INFO: `/v3/projects/df068037811443fbbea3944d108df9be/users/c7cf3d34bc3446adb439b637fe1b1a2b/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.562503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6329-667f-4dfd-80f5-0578f662f9ca admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.563748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b327f-177a-4854-b75f-5d7682d372e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df068037811443fbbea3944d108df9be, user_id=c7cf3d34bc3446adb439b637fe1b1a2b, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.566779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b327f-177a-4854-b75f-5d7682d372e5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.567476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.568144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.568419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.568688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.569464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.571135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.580586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 807/1600] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:12 2022] PUT /identity/v3/projects/df068037811443fbbea3944d108df9be/users/c7cf3d34bc3446adb439b637fe1b1a2b/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:12.581963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ce45-aa87-4a71-9963-50b9f1156024 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.582307 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ce45-aa87-4a71-9963-50b9f1156024 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.582689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693ce45-aa87-4a71-9963-50b9f1156024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.583483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 794/1601] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.584391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.603558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c43067-6f8b-4a07-9f5a-3010d473aebd admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.605417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8693ce45-aa87-4a71-9963-50b9f1156024 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:12.607198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8693ce45-aa87-4a71-9963-50b9f1156024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:12.674384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7274e-8251-42c0-9e8a-6e273580c866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.675069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7274e-8251-42c0-9e8a-6e273580c866 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.675349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7274e-8251-42c0-9e8a-6e273580c866 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.675619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7274e-8251-42c0-9e8a-6e273580c866 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.676357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7274e-8251-42c0-9e8a-6e273580c866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.678053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7274e-8251-42c0-9e8a-6e273580c866 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.691225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 795/1602] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.694556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea84bab-63b9-4360-b5d1-36647b3314f3 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.694945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea84bab-63b9-4360-b5d1-36647b3314f3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.695316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea84bab-63b9-4360-b5d1-36647b3314f3 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.697249 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 796/1603] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 4 14:52:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:12.699795 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 808/1604] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:12.701418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ce45-aa87-4a71-9963-50b9f1156024 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.703393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea84bab-63b9-4360-b5d1-36647b3314f3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.715506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693ce45-aa87-4a71-9963-50b9f1156024 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.717753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea84bab-63b9-4360-b5d1-36647b3314f3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.721469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.722400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.722749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.723201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.723390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.724155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.724333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.724502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.724694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] PATH_INFO: `/v3/projects/35d8071dbd7246228c7a252c40fae038/users/aff6489e5a6441509376c0dca551f03c/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.726351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.743434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 809/1605] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.743434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.744998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d8071dbd7246228c7a252c40fae038, user_id=aff6489e5a6441509376c0dca551f03c, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.750057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.758665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414b7f1-5ef7-450d-b011-8ed14afc27d1 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.766432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe279ee-2e8a-4ebd-926e-2f1c85a8f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.767296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe279ee-2e8a-4ebd-926e-2f1c85a8f0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.767684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe279ee-2e8a-4ebd-926e-2f1c85a8f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.768057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe279ee-2e8a-4ebd-926e-2f1c85a8f0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.768975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe279ee-2e8a-4ebd-926e-2f1c85a8f0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.770747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe279ee-2e8a-4ebd-926e-2f1c85a8f0b2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.775458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 797/1606] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:12 2022] PUT /identity/v3/projects/35d8071dbd7246228c7a252c40fae038/users/aff6489e5a6441509376c0dca551f03c/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:12.775458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.781545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 810/1607] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.783019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7774451-7945-4f63-9468-05194ab28e4a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.783306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7774451-7945-4f63-9468-05194ab28e4a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.783579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7774451-7945-4f63-9468-05194ab28e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.829955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a7774451-7945-4f63-9468-05194ab28e4a None None] MFA Rules not processed for user `c7cf3d34bc3446adb439b637fe1b1a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:12.831514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a7774451-7945-4f63-9468-05194ab28e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:12.854861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9192435-83aa-4c8b-9dd2-e30ba8c9a3d0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.860099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.861073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.861357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.861631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] PATH_INFO: `/v3/projects/f512333663064960847a1dd30c177cf9/users/da47bf4c34f243bb9493d8bc11891cef/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.877207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f512333663064960847a1dd30c177cf9, user_id=da47bf4c34f243bb9493d8bc11891cef, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.879939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.896905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 798/1608] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:12 2022] PUT /identity/v3/projects/f512333663064960847a1dd30c177cf9/users/da47bf4c34f243bb9493d8bc11891cef/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:12.901330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:12.933379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 811/1609] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:12.934835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e0b5-f6c5-46b8-8ba9-93de30b0e492 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.935184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e0b5-f6c5-46b8-8ba9-93de30b0e492 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.935791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e0b5-f6c5-46b8-8ba9-93de30b0e492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.965096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1b69e0b5-f6c5-46b8-8ba9-93de30b0e492 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:12.966685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1b69e0b5-f6c5-46b8-8ba9-93de30b0e492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:12.968312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092e37-1325-40cd-b73f-cbb5cab0fd1f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:12.974306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:12.974998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:12.975278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:12.975546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:12.976292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:12.977994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:12.990082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 799/1610] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:12.990996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.046359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 812/1611] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 4 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:13.048122 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e0b5-f6c5-46b8-8ba9-93de30b0e492 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.059935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e0b5-f6c5-46b8-8ba9-93de30b0e492 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.065489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58bed5-fa0f-4bcf-88b4-edb3f89c9651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.066838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58bed5-fa0f-4bcf-88b4-edb3f89c9651 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.067304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58bed5-fa0f-4bcf-88b4-edb3f89c9651 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.067780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58bed5-fa0f-4bcf-88b4-edb3f89c9651 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.069176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58bed5-fa0f-4bcf-88b4-edb3f89c9651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.072299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58bed5-fa0f-4bcf-88b4-edb3f89c9651 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.072943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb24cec-f952-4595-99df-0e1c8d400eed admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.077075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.077771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.078086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.078343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.088942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.090430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.091786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 813/1612] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:13.094042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f9c2d-0363-437f-be63-9b5141422909 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.094598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f9c2d-0363-437f-be63-9b5141422909 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.095099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f9c2d-0363-437f-be63-9b5141422909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.102905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 800/1613] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:13.103854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.114729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3146ee4-17e8-43d7-b331-1015e0464f60 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.120620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.121641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.121921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.122205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] PATH_INFO: `/v3/projects/c94f284eff2d4c5298d628d63d6086c7/users/f908f77e75f44b3ca5f30f4f04395ece/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.127152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1f7f9c2d-0363-437f-be63-9b5141422909 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:13.129492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1f7f9c2d-0363-437f-be63-9b5141422909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:13.142702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94f284eff2d4c5298d628d63d6086c7, user_id=f908f77e75f44b3ca5f30f4f04395ece, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.144299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 814/1614] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:13.146990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9c463-e3bf-45b7-b178-917062863585 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.147480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9c463-e3bf-45b7-b178-917062863585 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.148106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9c463-e3bf-45b7-b178-917062863585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.149560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.169772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 801/1615] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:13 2022] PUT /identity/v3/projects/c94f284eff2d4c5298d628d63d6086c7/users/f908f77e75f44b3ca5f30f4f04395ece/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:13.171699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.179174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-53a9c463-e3bf-45b7-b178-917062863585 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:13.181080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-53a9c463-e3bf-45b7-b178-917062863585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:13.284297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ee43a-1e38-46c0-9f7f-fd945a8ca373 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.290513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 815/1616] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:13.292681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.294910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9c463-e3bf-45b7-b178-917062863585 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.297118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.298710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.299519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.307031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9c463-e3bf-45b7-b178-917062863585 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.313468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.314650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.314955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.315234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] PATH_INFO: `/v3/projects/f96454a48c304aabbcb82ad8a2771b01/users/06d696a14671493db2c243300a19cbc0/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.340902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96454a48c304aabbcb82ad8a2771b01, user_id=06d696a14671493db2c243300a19cbc0, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.344040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.369072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 816/1617] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:13 2022] PUT /identity/v3/projects/f96454a48c304aabbcb82ad8a2771b01/users/06d696a14671493db2c243300a19cbc0/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:13.369072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.424384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.428446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.455447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 802/1618] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:13.456744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.475901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb359420-ab6b-48e1-937f-8fa39aeec317 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.495914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e797d-4caa-4286-92fb-62a2e5e12c97 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.502799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.503747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.504066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.504446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.572173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f2ccc-6e14-4035-ac03-ce43ce37cbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.573968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f2ccc-6e14-4035-ac03-ce43ce37cbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.574711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f2ccc-6e14-4035-ac03-ce43ce37cbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.575238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f2ccc-6e14-4035-ac03-ce43ce37cbc8 admin admin] PATH_INFO: `/v3/projects/367321a836de445f9e65af1baac99c98/users/a0944ce1e8504fb1b50507ace4e63649/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.602560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.603854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f2ccc-6e14-4035-ac03-ce43ce37cbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=367321a836de445f9e65af1baac99c98, user_id=a0944ce1e8504fb1b50507ace4e63649, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.604644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.609331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f2ccc-6e14-4035-ac03-ce43ce37cbc8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.615517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 817/1619] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:13.616368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.632920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 803/1620] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:13 2022] PUT /identity/v3/projects/367321a836de445f9e65af1baac99c98/users/a0944ce1e8504fb1b50507ace4e63649/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:13.635723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ed6ee-48ce-41b2-a338-de4cd07346dd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.636246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ed6ee-48ce-41b2-a338-de4cd07346dd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.636960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ed6ee-48ce-41b2-a338-de4cd07346dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.686600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19fa8c-f4f6-4616-b0b2-f03abb98a44e service nova] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.727170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3e8ed6ee-48ce-41b2-a338-de4cd07346dd None None] MFA Rules not processed for user `f908f77e75f44b3ca5f30f4f04395ece`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:13.730889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3e8ed6ee-48ce-41b2-a338-de4cd07346dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:13.746571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e83df-6f9f-4a35-9934-c0f9be3f632f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.747469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e83df-6f9f-4a35-9934-c0f9be3f632f service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.748162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e83df-6f9f-4a35-9934-c0f9be3f632f service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.748582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e83df-6f9f-4a35-9934-c0f9be3f632f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.851940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e83df-6f9f-4a35-9934-c0f9be3f632f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.856121 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e83df-6f9f-4a35-9934-c0f9be3f632f service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.877092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 804/1621] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:13.879422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ed6ee-48ce-41b2-a338-de4cd07346dd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.882080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 818/1622] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:13.883484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441bb56-70fd-4c69-bd87-e704359e7366 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.883770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441bb56-70fd-4c69-bd87-e704359e7366 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.884048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441bb56-70fd-4c69-bd87-e704359e7366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.902752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ed6ee-48ce-41b2-a338-de4cd07346dd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:13.907220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0441bb56-70fd-4c69-bd87-e704359e7366 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:13.908722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0441bb56-70fd-4c69-bd87-e704359e7366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:13.911690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f65311-f2e9-4843-937a-e6338b3ad1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:13.913095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65311-f2e9-4843-937a-e6338b3ad1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.913346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65311-f2e9-4843-937a-e6338b3ad1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.913618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f65311-f2e9-4843-937a-e6338b3ad1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.920852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 819/1623] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:13.920852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441bb56-70fd-4c69-bd87-e704359e7366 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:13.936347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f65311-f2e9-4843-937a-e6338b3ad1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:13.939970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f65311-f2e9-4843-937a-e6338b3ad1b4 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:13.961554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 805/1624] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:13.964196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc880ba-60e0-40c2-85e0-9236fd40ceb4 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:13.965134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc880ba-60e0-40c2-85e0-9236fd40ceb4 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:13.965314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc880ba-60e0-40c2-85e0-9236fd40ceb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:13.997716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6cc880ba-60e0-40c2-85e0-9236fd40ceb4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:13.999720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441bb56-70fd-4c69-bd87-e704359e7366 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.001062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6cc880ba-60e0-40c2-85e0-9236fd40ceb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:14.013651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.014344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.014642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.015227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.019942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 806/1625] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:14.022012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f8681-e848-493c-93f2-6026f5bb9e47 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.022442 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f8681-e848-493c-93f2-6026f5bb9e47 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.023004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f8681-e848-493c-93f2-6026f5bb9e47 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.025150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 807/1626] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 4 14:52:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:14.025715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f8681-e848-493c-93f2-6026f5bb9e47 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:14.047826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f8681-e848-493c-93f2-6026f5bb9e47 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.055349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.056221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.056524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.056962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.057886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:14.059932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:14.105799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 808/1627] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:14 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:14.105799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:14.146053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:14.149023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:14.184528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 820/1628] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:14.186722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:14.196643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dcdf34-2f5a-4380-a866-420da509dee6 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.202327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71362b-40a7-4de6-82fa-4845ea22bc8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.203148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71362b-40a7-4de6-82fa-4845ea22bc8a service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.203699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71362b-40a7-4de6-82fa-4845ea22bc8a service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.204074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71362b-40a7-4de6-82fa-4845ea22bc8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.208880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62870733-ab1c-4064-bf95-cf2a03cdb01b service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.216487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.218132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.218724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.219494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.303958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71362b-40a7-4de6-82fa-4845ea22bc8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:14.310115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71362b-40a7-4de6-82fa-4845ea22bc8a service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:14.344304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 809/1629] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:14.347712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075420fe-5b7f-489b-b4d6-36c4d7d179b8 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.348208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075420fe-5b7f-489b-b4d6-36c4d7d179b8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.348698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075420fe-5b7f-489b-b4d6-36c4d7d179b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.384050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:14.385972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:14.403994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-075420fe-5b7f-489b-b4d6-36c4d7d179b8 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:14.406784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-075420fe-5b7f-489b-b4d6-36c4d7d179b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:14.415118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 821/1630] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:14.416082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:14.434482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105423b3-b4ad-4c99-aa5b-6ff1aee3cf06 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.450012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.450721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.451002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.451274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.451997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:14.454441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:14.499708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 822/1631] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:14 2022] POST /identity/v3/users => generated 289 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:14.505219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:14.555657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94be35-c2ad-412b-89b6-9ba8795c9bb7 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.568192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.569048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.569346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.569630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.570721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:14.572764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:14.596906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 823/1632] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:14.598866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:14.623919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 810/1633] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:14.664134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075420fe-5b7f-489b-b4d6-36c4d7d179b8 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:14.670589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f35a7d-ddb7-4a6d-b71d-fe9792b4c23c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.689473 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2085e2f-6ee3-4ac1-9f60-efd20c98beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.690424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2085e2f-6ee3-4ac1-9f60-efd20c98beb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.690861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2085e2f-6ee3-4ac1-9f60-efd20c98beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.691144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2085e2f-6ee3-4ac1-9f60-efd20c98beb9 admin admin] PATH_INFO: `/v3/projects/5533114b16844f279d845323e9cc12a7/users/95e85dcd45424cb7999a22ec50d11739/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.737624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2085e2f-6ee3-4ac1-9f60-efd20c98beb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5533114b16844f279d845323e9cc12a7, user_id=95e85dcd45424cb7999a22ec50d11739, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:14.740662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2085e2f-6ee3-4ac1-9f60-efd20c98beb9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:14.778815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 824/1634] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:14 2022] PUT /identity/v3/projects/5533114b16844f279d845323e9cc12a7/users/95e85dcd45424cb7999a22ec50d11739/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:14.778815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56852-a3a5-4512-9322-08c6d4e61b2c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.778815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56852-a3a5-4512-9322-08c6d4e61b2c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.778815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d56852-a3a5-4512-9322-08c6d4e61b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:14.804047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075420fe-5b7f-489b-b4d6-36c4d7d179b8 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:14.849747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-61d56852-a3a5-4512-9322-08c6d4e61b2c None None] MFA Rules not processed for user `b4a70ea6369348bea63dc3eac7ac0380`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:14.852476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-61d56852-a3a5-4512-9322-08c6d4e61b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:14.947176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782beb4-b9b2-4e19-81da-9aaa0a93ff4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:14.948033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782beb4-b9b2-4e19-81da-9aaa0a93ff4f service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:14.948580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782beb4-b9b2-4e19-81da-9aaa0a93ff4f service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:14.949152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782beb4-b9b2-4e19-81da-9aaa0a93ff4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.004311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 825/1635] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:15.005438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d56852-a3a5-4512-9322-08c6d4e61b2c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:15.071323 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782beb4-b9b2-4e19-81da-9aaa0a93ff4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:15.073749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782beb4-b9b2-4e19-81da-9aaa0a93ff4f service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:15.092124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 811/1636] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:15.093944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6301dc-cb33-4c13-b452-092fcee63724 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.094456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6301dc-cb33-4c13-b452-092fcee63724 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.094983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6301dc-cb33-4c13-b452-092fcee63724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.121861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-cf6301dc-cb33-4c13-b452-092fcee63724 None None] MFA Rules not processed for user `b4a70ea6369348bea63dc3eac7ac0380`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:15.123776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cf6301dc-cb33-4c13-b452-092fcee63724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:15.142034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 812/1637] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:15.189703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd147d8-1343-4be6-8760-fdf441fd343e None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.190180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd147d8-1343-4be6-8760-fdf441fd343e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.190847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd147d8-1343-4be6-8760-fdf441fd343e None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.197187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 813/1638] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 4 14:52:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:15.204627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d56852-a3a5-4512-9322-08c6d4e61b2c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:15.206766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3af57-d311-490d-94d9-252d5c70ae02 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.207173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3af57-d311-490d-94d9-252d5c70ae02 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.207572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3af57-d311-490d-94d9-252d5c70ae02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.237448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:15.238047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.238547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.239024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.240418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:15.243629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:15.272485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-add3af57-d311-490d-94d9-252d5c70ae02 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:15.274690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-add3af57-d311-490d-94d9-252d5c70ae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:15.287769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 826/1639] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:15.296617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:15.300050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 814/1640] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 4 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:15.330600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de78848-ffa9-45d9-a999-ad59c1dfa627 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:15.343304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3af57-d311-490d-94d9-252d5c70ae02 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:15.348448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:15.350054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.350547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.351022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] PATH_INFO: `/v3/projects/5533114b16844f279d845323e9cc12a7/users/95e85dcd45424cb7999a22ec50d11739/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.362503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3af57-d311-490d-94d9-252d5c70ae02 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:15.367496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5533114b16844f279d845323e9cc12a7, user_id=95e85dcd45424cb7999a22ec50d11739, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:15.373216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:15.376630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad8ed9-13aa-4b70-9706-72632dead8e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:15.377777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad8ed9-13aa-4b70-9706-72632dead8e5 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.378889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad8ed9-13aa-4b70-9706-72632dead8e5 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.379361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad8ed9-13aa-4b70-9706-72632dead8e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.390899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad8ed9-13aa-4b70-9706-72632dead8e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:15.394762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad8ed9-13aa-4b70-9706-72632dead8e5 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:15.408032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 815/1641] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:15.423240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 827/1642] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:15 2022] PUT /identity/v3/projects/5533114b16844f279d845323e9cc12a7/users/95e85dcd45424cb7999a22ec50d11739/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:15.425939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa58c5c-8204-490d-8f2a-9f69a0e5299b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.426499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa58c5c-8204-490d-8f2a-9f69a0e5299b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.426959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa58c5c-8204-490d-8f2a-9f69a0e5299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.503453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0fa58c5c-8204-490d-8f2a-9f69a0e5299b None None] MFA Rules not processed for user `95e85dcd45424cb7999a22ec50d11739`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:15.504762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0fa58c5c-8204-490d-8f2a-9f69a0e5299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:15.695756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 816/1643] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:15.710803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:15.896656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5a1f-61a6-4be3-9ff2-55bf36a4021a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:15.910667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6ee63-c7c5-4237-bf52-51b59b705a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:15.911383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6ee63-c7c5-4237-bf52-51b59b705a43 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:15.911666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6ee63-c7c5-4237-bf52-51b59b705a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:15.911942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6ee63-c7c5-4237-bf52-51b59b705a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:15.926156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6ee63-c7c5-4237-bf52-51b59b705a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:15.927660 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6ee63-c7c5-4237-bf52-51b59b705a43 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:15.963571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 828/1644] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:16.028960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadf61a-acf5-46f4-bac3-105156f52291 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:16.029555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadf61a-acf5-46f4-bac3-105156f52291 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:16.029994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadf61a-acf5-46f4-bac3-105156f52291 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:16.032321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 817/1645] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 4 14:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:16.039406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:16.040012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:16.040453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:16.072185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:16.081103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:16.403862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadf61a-acf5-46f4-bac3-105156f52291 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:16.416053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 829/1646] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:16.446796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:16.449299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:16.457815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadf61a-acf5-46f4-bac3-105156f52291 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:16.475852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9749cf-69c2-41ec-8e1a-2cc1b1151eb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:16.476947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9749cf-69c2-41ec-8e1a-2cc1b1151eb8 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:16.478353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9749cf-69c2-41ec-8e1a-2cc1b1151eb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:16.479067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9749cf-69c2-41ec-8e1a-2cc1b1151eb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:16.632886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee19e99-2b9e-4eb6-868a-2adca2694010 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:16.642615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ed3b-ebf2-4d40-887a-80613a2ba958 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:16.643715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ed3b-ebf2-4d40-887a-80613a2ba958 service glance] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:16.644195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ed3b-ebf2-4d40-887a-80613a2ba958 service glance] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:16.644660 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ed3b-ebf2-4d40-887a-80613a2ba958 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:16.673368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ed3b-ebf2-4d40-887a-80613a2ba958 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:16.675503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ed3b-ebf2-4d40-887a-80613a2ba958 service glance] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:16.705237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 830/1647] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:16.776114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9749cf-69c2-41ec-8e1a-2cc1b1151eb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:16.780640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9749cf-69c2-41ec-8e1a-2cc1b1151eb8 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:16.851672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 818/1648] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:17.897377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14227b60-52b4-474f-b273-71b06f910f1f None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:17.897377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14227b60-52b4-474f-b273-71b06f910f1f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:17.897377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14227b60-52b4-474f-b273-71b06f910f1f None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:17.900055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 831/1649] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 4 14:52:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:17.908454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9afb6-d2bd-477c-ac7e-8902c26ed397 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:17.909068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9afb6-d2bd-477c-ac7e-8902c26ed397 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:17.909068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9afb6-d2bd-477c-ac7e-8902c26ed397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:17.991669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a2b9afb6-d2bd-477c-ac7e-8902c26ed397 None None] MFA Rules not processed for user `0d678d5ee7d74896bbc1316b32ec81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:17.993747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a2b9afb6-d2bd-477c-ac7e-8902c26ed397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:18.146727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 819/1650] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:18.193551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14227b60-52b4-474f-b273-71b06f910f1f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:18.213671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14227b60-52b4-474f-b273-71b06f910f1f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:18.230069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000a6b3-c221-4f5c-b331-5210395d9966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:18.232332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000a6b3-c221-4f5c-b331-5210395d9966 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:18.232870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000a6b3-c221-4f5c-b331-5210395d9966 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:18.233289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000a6b3-c221-4f5c-b331-5210395d9966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:18.277040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000a6b3-c221-4f5c-b331-5210395d9966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:18.284182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000a6b3-c221-4f5c-b331-5210395d9966 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:18.334102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 832/1651] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:19.124548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9afb6-d2bd-477c-ac7e-8902c26ed397 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:19.275875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9afb6-d2bd-477c-ac7e-8902c26ed397 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:19.294512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:19.295724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:19.296094 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:19.296426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] PATH_INFO: `/v3/users/75c9f8cec68240e3a09dd4b8a95502bc` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:19.302383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c9f8cec68240e3a09dd4b8a95502bc)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:19.304462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: WARNING py.warnings [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:52:19.304462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: warnings.warn(msg) May 04 14:52:19.304462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: May 04 14:52:19.307021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:19.472702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c9f8cec68240e3a09dd4b8a95502bc'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:19.475650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c9f8cec68240e3a09dd4b8a95502bc'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:19.476661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c9f8cec68240e3a09dd4b8a95502bc'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:19.508957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c9f8cec68240e3a09dd4b8a95502bc'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:19.545085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c9f8cec68240e3a09dd4b8a95502bc'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:19.591720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 820/1652] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:19 2022] DELETE /identity/v3/users/75c9f8cec68240e3a09dd4b8a95502bc => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:19.592029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe4e3f-af7e-4a42-8419-198b742e5334 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:19.592542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe4e3f-af7e-4a42-8419-198b742e5334 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:19.592634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe4e3f-af7e-4a42-8419-198b742e5334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:19.630455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-16fe4e3f-af7e-4a42-8419-198b742e5334 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:19.632294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-16fe4e3f-af7e-4a42-8419-198b742e5334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:19.770454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 833/1653] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:19.855463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:19.859032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:20.144074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc12c12-22fd-431c-9018-8fb9e9ad1db3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:20.154746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540ae77-e76e-40bd-98d7-e4b82495b3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:20.160387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ae77-e76e-40bd-98d7-e4b82495b3ab service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:20.163420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ae77-e76e-40bd-98d7-e4b82495b3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:20.165489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540ae77-e76e-40bd-98d7-e4b82495b3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:20.205046 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540ae77-e76e-40bd-98d7-e4b82495b3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:20.213154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540ae77-e76e-40bd-98d7-e4b82495b3ab service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:20.259394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 821/1654] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:20.458801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31efde-7700-43a2-b369-63bdb4bf2535 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:20.459183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31efde-7700-43a2-b369-63bdb4bf2535 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:20.459376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31efde-7700-43a2-b369-63bdb4bf2535 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:20.460823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 834/1655] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 4 14:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:20.470012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:20.470012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:20.470012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:20.501120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] MFA Rules not processed for user `7ac21bca8da642338571f821ef5ea121`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:20.502383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:20.677818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 822/1656] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 4 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:20.689360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31efde-7700-43a2-b369-63bdb4bf2535 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:20.717585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31efde-7700-43a2-b369-63bdb4bf2535 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:20.743318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd493ce-69fc-49dc-ad8b-335fa910f21c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac21bca8da642338571f821ef5ea121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:20.744015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd493ce-69fc-49dc-ad8b-335fa910f21c service placement] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:20.744300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd493ce-69fc-49dc-ad8b-335fa910f21c service placement] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:20.744579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd493ce-69fc-49dc-ad8b-335fa910f21c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:20.749613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2cd493ce-69fc-49dc-ad8b-335fa910f21c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:20.786039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd493ce-69fc-49dc-ad8b-335fa910f21c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:20.790785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd493ce-69fc-49dc-ad8b-335fa910f21c service placement] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:20.824097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 835/1657] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:22.031183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:22.033418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:22.077059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f6daff-5f36-4b36-8870-2eed09d15bb3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:22.098946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:22.100248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:22.100672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:22.101100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] PATH_INFO: `/v3/users/f908f77e75f44b3ca5f30f4f04395ece` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:22.103809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f908f77e75f44b3ca5f30f4f04395ece)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:22.110577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:22.231745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f908f77e75f44b3ca5f30f4f04395ece'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.235379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f908f77e75f44b3ca5f30f4f04395ece'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.239651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f908f77e75f44b3ca5f30f4f04395ece'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.266870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f908f77e75f44b3ca5f30f4f04395ece'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.297359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f908f77e75f44b3ca5f30f4f04395ece'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.330497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 823/1658] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:22 2022] DELETE /identity/v3/users/f908f77e75f44b3ca5f30f4f04395ece => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:22.331969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:22.331969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:22.332626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:22.376905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:22.378458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:22.555733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 836/1659] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:22.593687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:22.596530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:22.656591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1aa6-b5d5-4d38-a6f5-4a3a381986fa admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:22.671362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:22.674270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:22.677518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:22.679896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] PATH_INFO: `/v3/projects/68496c30e63b44c3b89fb209c4160b0f` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:22.684552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] RBAC: Authorizing `identity:delete_project(project_id=68496c30e63b44c3b89fb209c4160b0f)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:22.686903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:22.747038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:22.748966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:22.818035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68496c30e63b44c3b89fb209c4160b0f', 'request_id': 'req-7127ed43-2bdd-421d-9095-14cdcc95810f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.863272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68496c30e63b44c3b89fb209c4160b0f', 'request_id': 'req-7127ed43-2bdd-421d-9095-14cdcc95810f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.880306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0883c-22f8-4874-8def-efa5c17836d0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:22.884566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68496c30e63b44c3b89fb209c4160b0f', 'request_id': 'req-7127ed43-2bdd-421d-9095-14cdcc95810f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.920304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] The token cache is being invalidate because project 68496c30e63b44c3b89fb209c4160b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:22.920493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.926663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7127ed43-2bdd-421d-9095-14cdcc95810f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:22.933532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 824/1660] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:22 2022] DELETE /identity/v3/projects/68496c30e63b44c3b89fb209c4160b0f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:22.939341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532591-ef42-4dbc-ab0e-b3835828f4ce None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:22.940419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532591-ef42-4dbc-ab0e-b3835828f4ce None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:22.942785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16532591-ef42-4dbc-ab0e-b3835828f4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:22.968163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:22.969643 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:22.970522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:22.971430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:22.974516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:23.027379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-16532591-ef42-4dbc-ab0e-b3835828f4ce None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:23.028750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-16532591-ef42-4dbc-ab0e-b3835828f4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:23.133998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 825/1661] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:23.142651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4eeb1d-15dd-4a6b-bfbb-42c15a6e8eac None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.143314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4eeb1d-15dd-4a6b-bfbb-42c15a6e8eac None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.143864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4eeb1d-15dd-4a6b-bfbb-42c15a6e8eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:23.188521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ef4eeb1d-15dd-4a6b-bfbb-42c15a6e8eac None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:23.189645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:23.190269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ef4eeb1d-15dd-4a6b-bfbb-42c15a6e8eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:23.193844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:23.204630 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 826/1662] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:23.208590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4eeb1d-15dd-4a6b-bfbb-42c15a6e8eac None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:23.239882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 837/1663] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:23.240639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4eeb1d-15dd-4a6b-bfbb-42c15a6e8eac None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:23.249775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:23.250931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.251562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.252110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:23.253878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:23.255687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:23.262460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:23.301286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 827/1664] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:23.305602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:23.321835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba04c03-d599-4151-b3fc-169a0ccee4eb service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:23.332902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:23.335858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.336362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.336881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:23.337799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:23.341818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:23.395103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 838/1665] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:23 2022] POST /identity/v3/projects => generated 367 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:23.402103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:23.421065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc4691-1825-42b8-86ae-1ce23f1ae869 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:23.520727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:23.521485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.521772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.522052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:23.522836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:23.525521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:23.574168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 828/1666] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:23 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:23.577817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:23.600827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d032d-1e11-4010-883f-31da38a0a5e3 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:23.613381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:23.614103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.614385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.614657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:23.615429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:23.617277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:23.657378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 839/1667] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:23.658712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:23.675294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da673a2c-f5cf-4a05-a9ea-aa2aadedbae8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:23.694021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e81742-e6f1-4b7c-90f6-7ec2726a96e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:23.694738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e81742-e6f1-4b7c-90f6-7ec2726a96e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.695024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e81742-e6f1-4b7c-90f6-7ec2726a96e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.695302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e81742-e6f1-4b7c-90f6-7ec2726a96e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:23.696065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e81742-e6f1-4b7c-90f6-7ec2726a96e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:23.699113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e81742-e6f1-4b7c-90f6-7ec2726a96e9 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:23.776706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 829/1668] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:52:23.781427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:23.799517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4a6f6-d30a-4688-a5e8-dda01476763c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:23.815259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:23.815923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.816209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.816478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] PATH_INFO: `/v3/projects/b2e534493a0f4ca2802dd8db566ae72d/users/f650aaf27e0f4118a088f2b6714e6088/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:23.877161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e534493a0f4ca2802dd8db566ae72d, user_id=f650aaf27e0f4118a088f2b6714e6088, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:23.880897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:23.933048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 840/1669] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:23 2022] PUT /identity/v3/projects/b2e534493a0f4ca2802dd8db566ae72d/users/f650aaf27e0f4118a088f2b6714e6088/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:23.945540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaead52-78f4-429b-ba08-db2dec8e3d45 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:23.946076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaead52-78f4-429b-ba08-db2dec8e3d45 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:23.946522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaead52-78f4-429b-ba08-db2dec8e3d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:24.045851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-2eaead52-78f4-429b-ba08-db2dec8e3d45 None None] MFA Rules not processed for user `f650aaf27e0f4118a088f2b6714e6088`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:24.058922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2eaead52-78f4-429b-ba08-db2dec8e3d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:24.353568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 830/1670] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:24.359609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:24.602586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd61af-36a3-46f4-bcf7-5173012ae10d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:24.627773 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:24.629211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:24.629746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:24.630233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:24.631846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:24.638087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:24.696503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1d628-1756-413f-8d3e-1fe8d25766b0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:24.697074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1d628-1756-413f-8d3e-1fe8d25766b0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:24.697669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1d628-1756-413f-8d3e-1fe8d25766b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:24.736651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-81a1d628-1756-413f-8d3e-1fe8d25766b0 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:24.738223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-81a1d628-1756-413f-8d3e-1fe8d25766b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:24.748044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 841/1671] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:24 2022] POST /identity/v3/projects => generated 367 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:24.755733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:24.934076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 831/1672] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:25.144456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27fdf2-fa63-46ec-bb3c-6e534ba3f614 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:25.167974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:25.172831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:25.173339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:25.173633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:25.174413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:25.186239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:25.283142 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 842/1673] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:24 2022] POST /identity/v3/users => generated 290 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:25.289043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1d628-1756-413f-8d3e-1fe8d25766b0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:25.310266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1d628-1756-413f-8d3e-1fe8d25766b0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:25.336297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:25.339282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:25.341814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:25.342538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:25.344286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:25.350871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:25.403683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 832/1674] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:52:25.408880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:25.411088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:25.413289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:25.450462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989968f-1996-4074-8dbb-c1acd031bb42 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:25.483402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:25.485698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:25.486441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:25.487749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] PATH_INFO: `/v3/projects/8b09c1a142e649da8492e92e17ac227d/users/39a75456bead414f94ca92898076dce6/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:25.574881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b09c1a142e649da8492e92e17ac227d, user_id=39a75456bead414f94ca92898076dce6, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:25.579815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:25.668246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 833/1675] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:25 2022] PUT /identity/v3/projects/8b09c1a142e649da8492e92e17ac227d/users/39a75456bead414f94ca92898076dce6/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:25.670192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:25.793151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47165c2e-ce7d-469e-a048-ee4f3a8ccbf2 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:26.003302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448111bb-6c9d-4cd0-8525-ed527ec7fa2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:26.004176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448111bb-6c9d-4cd0-8525-ed527ec7fa2e service swift] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:26.004573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448111bb-6c9d-4cd0-8525-ed527ec7fa2e service swift] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:26.004886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448111bb-6c9d-4cd0-8525-ed527ec7fa2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:26.077883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72689641-af9f-462b-b648-e75c24e22848 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:26.106004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:26.107185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:26.107740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:26.108140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:26.109694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:26.133948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448111bb-6c9d-4cd0-8525-ed527ec7fa2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:26.136125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448111bb-6c9d-4cd0-8525-ed527ec7fa2e service swift] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:26.143014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:26.178243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 843/1676] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:52:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:52:26.180528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:26.181032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:26.181332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:26.193467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 834/1677] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 524 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:26.195050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:26.227869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351fc66-8d24-4c9e-98a8-bfaffba36dd7 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:26.269761 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3f447-9eb0-4238-a2b3-b79ca8e15e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:26.271531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f447-9eb0-4238-a2b3-b79ca8e15e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:26.271930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f447-9eb0-4238-a2b3-b79ca8e15e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:26.272077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] MFA Rules not processed for user `da47bf4c34f243bb9493d8bc11891cef`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:26.272414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f447-9eb0-4238-a2b3-b79ca8e15e43 admin admin] PATH_INFO: `/v3/projects/8b09c1a142e649da8492e92e17ac227d/users/39a75456bead414f94ca92898076dce6/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:26.273309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:26.297487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3f447-9eb0-4238-a2b3-b79ca8e15e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b09c1a142e649da8492e92e17ac227d, user_id=39a75456bead414f94ca92898076dce6, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:26.303147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3f447-9eb0-4238-a2b3-b79ca8e15e43 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:26.396943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 835/1678] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:26 2022] PUT /identity/v3/projects/8b09c1a142e649da8492e92e17ac227d/users/39a75456bead414f94ca92898076dce6/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:26.409290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:26.414847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:26.415627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:26.537261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] MFA Rules not processed for user `39a75456bead414f94ca92898076dce6`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:26.539201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:26.781113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 844/1679] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 603 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:26.793385 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 836/1680] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:26.826387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:26.830682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:26.849147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:26.851417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:27.234409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a45bb-8889-4a07-976d-aa30758d94ab None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:27.345466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad99ae-7ca6-4511-ba41-e2f33f779472 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:27.373025 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:27.374723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:27.375615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:27.376418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:27.506677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52146739-6323-449c-ba58-968614814f11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:27.515295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52146739-6323-449c-ba58-968614814f11 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:27.517130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52146739-6323-449c-ba58-968614814f11 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:27.518327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52146739-6323-449c-ba58-968614814f11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:27.560646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52146739-6323-449c-ba58-968614814f11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:27.576443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52146739-6323-449c-ba58-968614814f11 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:27.637003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 845/1681] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 812 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:27.653147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:27.656488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:27.697326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 837/1682] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 849 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:28.135620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52146739-6323-449c-ba58-968614814f11 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:28.139367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-52146739-6323-449c-ba58-968614814f11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:28.278981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52146739-6323-449c-ba58-968614814f11 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:28.300727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:28.301664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:28.301947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:28.302218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] PATH_INFO: `/v3/projects/c94f284eff2d4c5298d628d63d6086c7` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:28.304620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94f284eff2d4c5298d628d63d6086c7)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:28.305602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: WARNING py.warnings [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 04 14:52:28.305602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: warnings.warn(msg) May 04 14:52:28.305602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: May 04 14:52:28.306854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:28.608564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94f284eff2d4c5298d628d63d6086c7', 'request_id': 'req-cba25638-40ec-4e4a-8ca3-939588c8bcc8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:28.653257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94f284eff2d4c5298d628d63d6086c7', 'request_id': 'req-cba25638-40ec-4e4a-8ca3-939588c8bcc8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:28.692441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94f284eff2d4c5298d628d63d6086c7', 'request_id': 'req-cba25638-40ec-4e4a-8ca3-939588c8bcc8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:28.833001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] The token cache is being invalidate because project c94f284eff2d4c5298d628d63d6086c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:28.835414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:28.839842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:28.846309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 846/1683] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:28 2022] DELETE /identity/v3/projects/c94f284eff2d4c5298d628d63d6086c7 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:28.851008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:28.872512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f744406-908a-4e60-bc04-d6b970c3fb60 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:29.100003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:29.104602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:29.105749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:29.106156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] PATH_INFO: `/v3/users/95e85dcd45424cb7999a22ec50d11739` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:29.112685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95e85dcd45424cb7999a22ec50d11739)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:29.122697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:29.153264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:29.155546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:29.192528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba25638-40ec-4e4a-8ca3-939588c8bcc8 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:29.210322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:29.212953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:29.213721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:29.214994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] PATH_INFO: `/v3/users/366e2408410f4c2c930881f42c2b18c4` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:29.219257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=366e2408410f4c2c930881f42c2b18c4)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:29.226760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:29.266079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95e85dcd45424cb7999a22ec50d11739'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.266994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95e85dcd45424cb7999a22ec50d11739'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.267967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95e85dcd45424cb7999a22ec50d11739'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.308593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95e85dcd45424cb7999a22ec50d11739'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.348109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-13771e34-7003-434a-b8a2-2a4bbf72fe44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95e85dcd45424cb7999a22ec50d11739'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.392013 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 838/1684] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:28 2022] DELETE /identity/v3/users/95e85dcd45424cb7999a22ec50d11739 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:29.443340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366e2408410f4c2c930881f42c2b18c4'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.482449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366e2408410f4c2c930881f42c2b18c4'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.483970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366e2408410f4c2c930881f42c2b18c4'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.485663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366e2408410f4c2c930881f42c2b18c4'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.510694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0c70707-d962-4ab0-a480-6c754dc3dd48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366e2408410f4c2c930881f42c2b18c4'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:29.549761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 847/1685] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:29 2022] DELETE /identity/v3/users/366e2408410f4c2c930881f42c2b18c4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:29.556090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:29.556951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:29.557296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:29.596384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:29.600078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:29.635133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:29.636115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:29.636650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:29.750115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] MFA Rules not processed for user `0d678d5ee7d74896bbc1316b32ec81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:29.751538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:29.836166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 839/1686] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:29.954294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 848/1687] 10.222.0.49 () {58 vars in 1015 bytes} [Wed May 4 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:29.981864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:29.985759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:30.037734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8394769-eb5f-4207-8038-1114fbde0ca4 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:30.057711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:30.061813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:30.062168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:30.062474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:30.104836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:30.107196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:30.131742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 840/1688] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:30.164104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:30.166465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:30.197140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360af0d-5359-414d-8489-bd7e8baf0d1f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:30.217841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8831253-2215-4f85-9a32-ee5c6d43a950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:30.223869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8831253-2215-4f85-9a32-ee5c6d43a950 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:30.224404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8831253-2215-4f85-9a32-ee5c6d43a950 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:30.226233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8831253-2215-4f85-9a32-ee5c6d43a950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:30.268508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8831253-2215-4f85-9a32-ee5c6d43a950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:30.270247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8831253-2215-4f85-9a32-ee5c6d43a950 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:30.298232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 849/1689] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:31.568992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:31.571965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:31.598026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fc66b-9e86-4fc6-a1ea-1e135752ee26 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:31.607245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:31.608411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:31.609109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:31.609710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] PATH_INFO: `/v3/users/f650aaf27e0f4118a088f2b6714e6088` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:31.611438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f650aaf27e0f4118a088f2b6714e6088)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:31.614237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:31.703140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f650aaf27e0f4118a088f2b6714e6088'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:31.704186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f650aaf27e0f4118a088f2b6714e6088'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:31.705009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f650aaf27e0f4118a088f2b6714e6088'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:31.714674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f650aaf27e0f4118a088f2b6714e6088'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:31.730890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f650aaf27e0f4118a088f2b6714e6088'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:31.757860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 841/1690] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:31 2022] DELETE /identity/v3/users/f650aaf27e0f4118a088f2b6714e6088 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:31.765692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531874c-acf9-417a-8e67-3a71d4a1bf75 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:31.766450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531874c-acf9-417a-8e67-3a71d4a1bf75 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:31.766450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531874c-acf9-417a-8e67-3a71d4a1bf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:31.849043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6531874c-acf9-417a-8e67-3a71d4a1bf75 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:31.850993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6531874c-acf9-417a-8e67-3a71d4a1bf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:32.138799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 850/1691] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:32.632361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:32.634162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:32.798027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd8902-3b29-475e-ad0b-a0626691a536 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:32.804208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b4df8-f1a5-40e7-85a6-1d20e4fd704c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:32.805510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4df8-f1a5-40e7-85a6-1d20e4fd704c service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:32.805988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4df8-f1a5-40e7-85a6-1d20e4fd704c service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:32.806413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4df8-f1a5-40e7-85a6-1d20e4fd704c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:32.818435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b4df8-f1a5-40e7-85a6-1d20e4fd704c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:32.820517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b4df8-f1a5-40e7-85a6-1d20e4fd704c service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:32.867399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 842/1692] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:33.731557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e1b28-a963-4577-9aac-14ba279641fd None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:33.732677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e1b28-a963-4577-9aac-14ba279641fd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:33.733262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e1b28-a963-4577-9aac-14ba279641fd None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:33.735284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 851/1693] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:52:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:34.090840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:34.091511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:34.091791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:34.138059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:34.146810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:34.317725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 843/1694] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:52:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:34.374140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e1b28-a963-4577-9aac-14ba279641fd None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:34.376925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-965e1b28-a963-4577-9aac-14ba279641fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:34.420011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e1b28-a963-4577-9aac-14ba279641fd None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:34.436266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:34.437597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:34.437826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:34.438191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:34.465210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:34.467353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:34.507547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 852/1695] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:34.959274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:34.961065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:35.017910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d819ed-06a0-4021-8955-5daa27b243ba None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:35.032090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:35.036023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:35.037506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:35.037938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] PATH_INFO: `/v3/projects/9888acb2dc5d4b95881080cf62d40844` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:35.041825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9888acb2dc5d4b95881080cf62d40844)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:35.047480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:35.156883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:35.158581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:35.197685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9888acb2dc5d4b95881080cf62d40844', 'request_id': 'req-7c7857d8-0625-42b6-b365-8ecc136691eb'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.200816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21ded5-eb8c-4fa1-aa4a-8228e189b623 service nova] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:35.216689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9888acb2dc5d4b95881080cf62d40844', 'request_id': 'req-7c7857d8-0625-42b6-b365-8ecc136691eb'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.258275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9888acb2dc5d4b95881080cf62d40844', 'request_id': 'req-7c7857d8-0625-42b6-b365-8ecc136691eb'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.296311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] The token cache is being invalidate because project 9888acb2dc5d4b95881080cf62d40844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:35.296860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.317241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.319318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 844/1696] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:34 2022] DELETE /identity/v3/projects/9888acb2dc5d4b95881080cf62d40844 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:35.323460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:35.325512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:35.410200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:35.411380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:35.411661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:35.412020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] PATH_INFO: `/v3/projects/5533114b16844f279d845323e9cc12a7` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:35.413499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5533114b16844f279d845323e9cc12a7)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:35.416637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:35.502815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7857d8-0625-42b6-b365-8ecc136691eb admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:35.569748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5533114b16844f279d845323e9cc12a7', 'request_id': 'req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.585978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5533114b16844f279d845323e9cc12a7', 'request_id': 'req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.625832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:35.626915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:35.627058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5533114b16844f279d845323e9cc12a7', 'request_id': 'req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.627189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:35.627468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] PATH_INFO: `/v3/users/c7cf3d34bc3446adb439b637fe1b1a2b` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:35.640130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7cf3d34bc3446adb439b637fe1b1a2b)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:35.642577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:35.703434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] The token cache is being invalidate because project 5533114b16844f279d845323e9cc12a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:35.705037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.706522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-df4b1845-328e-4e0f-b6da-46e65ffd5a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.712863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 853/1697] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:35 2022] DELETE /identity/v3/projects/5533114b16844f279d845323e9cc12a7 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:35.721430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655b587-b433-4917-aa30-f599dae28aa3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:35.727594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655b587-b433-4917-aa30-f599dae28aa3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:35.730661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655b587-b433-4917-aa30-f599dae28aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:35.809595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7cf3d34bc3446adb439b637fe1b1a2b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.810773 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7cf3d34bc3446adb439b637fe1b1a2b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.812884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7cf3d34bc3446adb439b637fe1b1a2b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.833765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7cf3d34bc3446adb439b637fe1b1a2b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.896357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1861c746-09f3-4c5f-bef6-6902adf588c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7cf3d34bc3446adb439b637fe1b1a2b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:35.928342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 845/1698] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:35 2022] DELETE /identity/v3/users/c7cf3d34bc3446adb439b637fe1b1a2b => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:36.313527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6655b587-b433-4917-aa30-f599dae28aa3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:36.323901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6655b587-b433-4917-aa30-f599dae28aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:36.554756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 854/1699] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 838 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:36.564401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c975-ff85-4e1f-b35e-af8b43a8f4eb None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:36.565186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c975-ff85-4e1f-b35e-af8b43a8f4eb None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:36.565487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c975-ff85-4e1f-b35e-af8b43a8f4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:36.601769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b091c975-ff85-4e1f-b35e-af8b43a8f4eb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:36.604978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b091c975-ff85-4e1f-b35e-af8b43a8f4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:36.621666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655b587-b433-4917-aa30-f599dae28aa3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:36.629866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 846/1700] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:36.644371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655b587-b433-4917-aa30-f599dae28aa3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:36.651533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:36.652594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:36.653062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:36.653575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:36.654699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:36.655289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:36.657826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:36.680837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 855/1701] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:52:36.683387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c975-ff85-4e1f-b35e-af8b43a8f4eb None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:36.699451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c975-ff85-4e1f-b35e-af8b43a8f4eb None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:36.722102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:36.723691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:36.724367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:36.724821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:36.725819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:36.737779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:36.827043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:36.836056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 847/1702] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:36 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:36.841291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b1e1-20cc-44fa-a6a0-9954df4b133c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:36.956069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:36.956918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:36.958938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:36.959470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:36.962376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:36.969819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:37.052482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:37.060098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 856/1703] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:36 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:52:37.070153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3aa72-a534-487f-b7a9-5fd65ec541a2 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:37.080005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:37.081489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:37.081957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:37.082427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:37.083706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:37.088884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:37.113059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 848/1704] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:52:37.116911 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:37.133619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c058cf-ae74-46b3-99f6-8f40becdfe38 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:37.160855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:37.162113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:37.162526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:37.162903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:37.163980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:37.170015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:37.187133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 857/1705] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:52:37.191730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:37.220167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f915f2-48c2-46d6-a7fc-0186db0a8fcb admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:37.242516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1a7fa-7628-427e-9efc-5063cddf0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:37.245266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a7fa-7628-427e-9efc-5063cddf0851 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:37.247178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a7fa-7628-427e-9efc-5063cddf0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:37.247597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a7fa-7628-427e-9efc-5063cddf0851 admin admin] PATH_INFO: `/v3/projects/a8657b6e1ba1460987724472f11a00e4/users/5ac66da619424e6f8ce6d1c45afb376b/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:37.299381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1a7fa-7628-427e-9efc-5063cddf0851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8657b6e1ba1460987724472f11a00e4, user_id=5ac66da619424e6f8ce6d1c45afb376b, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:37.302975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1a7fa-7628-427e-9efc-5063cddf0851 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:37.326142 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:37.330659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:37.347575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 849/1706] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:37 2022] PUT /identity/v3/projects/a8657b6e1ba1460987724472f11a00e4/users/5ac66da619424e6f8ce6d1c45afb376b/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:37.363378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909e86-bbe1-4e83-b690-2546b99faad3 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:37.364099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909e86-bbe1-4e83-b690-2546b99faad3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:37.365673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909e86-bbe1-4e83-b690-2546b99faad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:37.430889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc1ac4-be26-4d2a-aeb5-5082790f19ef admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:37.451384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-03909e86-bbe1-4e83-b690-2546b99faad3 None None] MFA Rules not processed for user `5ac66da619424e6f8ce6d1c45afb376b`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:37.457737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-03909e86-bbe1-4e83-b690-2546b99faad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:37.621286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 850/1707] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:37.626186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03909e86-bbe1-4e83-b690-2546b99faad3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:37.695111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:37.719411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:37.721278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:37.722093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] PATH_INFO: `/v3/projects/b2e534493a0f4ca2802dd8db566ae72d` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:37.733998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e534493a0f4ca2802dd8db566ae72d)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:37.741852 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:37.787594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03909e86-bbe1-4e83-b690-2546b99faad3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:37.805596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:37.806338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:37.806631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:37.806914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:37.807674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:37.809980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:37.874418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 851/1708] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:37 2022] POST /identity/v3/projects => generated 385 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:52:37.879613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:37.980281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e534493a0f4ca2802dd8db566ae72d', 'request_id': 'req-c8b452fd-516f-4569-b684-933a7d46b387'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.008502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e534493a0f4ca2802dd8db566ae72d', 'request_id': 'req-c8b452fd-516f-4569-b684-933a7d46b387'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.042212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e534493a0f4ca2802dd8db566ae72d', 'request_id': 'req-c8b452fd-516f-4569-b684-933a7d46b387'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.080430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] The token cache is being invalidate because project b2e534493a0f4ca2802dd8db566ae72d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:38.082454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92905b-ed8c-40bc-bb4f-1c15b647058e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:38.082944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.084730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.088844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 858/1709] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:37 2022] DELETE /identity/v3/projects/b2e534493a0f4ca2802dd8db566ae72d => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:38.093334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:38.095168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:38.337198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b452fd-516f-4569-b684-933a7d46b387 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:38.367715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:38.371553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:38.373454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:38.374031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] PATH_INFO: `/v3/users/39a75456bead414f94ca92898076dce6` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:38.378098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a75456bead414f94ca92898076dce6)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:38.384691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:38.399768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:38.401031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:38.401559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:38.402418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:38.406659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:38.433192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:38.539827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 852/1710] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:37 2022] POST /identity/v3/users => generated 299 bytes in 661 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:38.543384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:38.573596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:38.595908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a75456bead414f94ca92898076dce6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.655877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1b29e-4004-47b4-9a16-c4244a48f8db admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:38.660778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a75456bead414f94ca92898076dce6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.664441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a75456bead414f94ca92898076dce6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.670072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a75456bead414f94ca92898076dce6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.688590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:38.693436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:38.695742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:38.697013 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] PATH_INFO: `/v3/projects/df068037811443fbbea3944d108df9be` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:38.703098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a75456bead414f94ca92898076dce6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.707693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df068037811443fbbea3944d108df9be)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:38.722035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:38.784857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 859/1711] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:38 2022] DELETE /identity/v3/users/39a75456bead414f94ca92898076dce6 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:38.789235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:38.797413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:38.911284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df068037811443fbbea3944d108df9be', 'request_id': 'req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.946845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df068037811443fbbea3944d108df9be', 'request_id': 'req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:38.968681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df068037811443fbbea3944d108df9be', 'request_id': 'req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:39.008153 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4772-ecc0-4bfe-8f14-609e32a53317 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:39.009084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] The token cache is being invalidate because project df068037811443fbbea3944d108df9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:39.011781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:39.016118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eaabc6f0-70cc-4a45-a7e2-96d9111a2e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:39.018842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 853/1712] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:38 2022] DELETE /identity/v3/projects/df068037811443fbbea3944d108df9be => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:39.024027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790973-146a-4b8b-8341-79c1a8619bf8 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.024292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790973-146a-4b8b-8341-79c1a8619bf8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.024609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee790973-146a-4b8b-8341-79c1a8619bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.092834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ee790973-146a-4b8b-8341-79c1a8619bf8 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:39.096014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ee790973-146a-4b8b-8341-79c1a8619bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:39.227084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:39.228110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.228550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.228992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.229927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:39.235632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:39.263828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 860/1713] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:39.268611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:39.270651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:39.288308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91f2e0-7307-46f0-be1c-ec286e380942 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:39.295823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:39.297558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.298018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.298472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] PATH_INFO: `/v3/projects/ae7406b07f984d8bb1f70d5fb7022f8f/users/ac8354829c934761a5f4128855130937/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.322579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7406b07f984d8bb1f70d5fb7022f8f, user_id=ac8354829c934761a5f4128855130937, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:39.326097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:39.334409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 854/1714] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:39.340916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2013b-ab87-4c73-9f88-e1d96a58557f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.343674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2013b-ab87-4c73-9f88-e1d96a58557f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.344898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2013b-ab87-4c73-9f88-e1d96a58557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.356402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 861/1715] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:39 2022] PUT /identity/v3/projects/ae7406b07f984d8bb1f70d5fb7022f8f/users/ac8354829c934761a5f4128855130937/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:39.357939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:39.416489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-48d2013b-ab87-4c73-9f88-e1d96a58557f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:39.418833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-48d2013b-ab87-4c73-9f88-e1d96a58557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:39.572492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3dd54-0665-4b3a-854e-73f55460a141 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:39.583380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:39.586030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.586321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.587621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.589344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:39.594931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:39.617634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 855/1716] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:39.624769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 862/1717] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:39.629681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2013b-ab87-4c73-9f88-e1d96a58557f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:39.634650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:39.667203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2013b-ab87-4c73-9f88-e1d96a58557f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:39.671398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a117be-a41b-4fde-8100-e209ae0831dc admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:39.680136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:39.680928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.681231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.681518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.682308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:39.690791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:39.693950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:39.696538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26131c-b5be-40dc-a658-19b214819d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:39.697589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26131c-b5be-40dc-a658-19b214819d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.697936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26131c-b5be-40dc-a658-19b214819d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.698280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26131c-b5be-40dc-a658-19b214819d32 admin admin] PATH_INFO: `/v3/projects/ae7406b07f984d8bb1f70d5fb7022f8f/users/ac8354829c934761a5f4128855130937/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.704186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26131c-b5be-40dc-a658-19b214819d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7406b07f984d8bb1f70d5fb7022f8f, user_id=ac8354829c934761a5f4128855130937, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:39.708219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26131c-b5be-40dc-a658-19b214819d32 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:39.733569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 856/1718] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:39.734709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:39.755138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 863/1719] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:39 2022] PUT /identity/v3/projects/ae7406b07f984d8bb1f70d5fb7022f8f/users/ac8354829c934761a5f4128855130937/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:39.759817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.760137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.760966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.773409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeef69a-9e36-49a7-ba62-d307d0eed661 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:39.837489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] MFA Rules not processed for user `ac8354829c934761a5f4128855130937`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:39.841557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:39.932569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:39.933351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:39.933633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:39.933955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:39.934682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:39.939990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:40.025741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 857/1720] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:39 2022] POST /identity/v3/projects => generated 365 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:40.029841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:40.081299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 864/1721] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:40.099057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:40.102030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:40.254777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3f431-834c-486f-acae-34f567520184 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:40.278104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:40.279283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:40.279607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:40.279857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:40.282246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:40.294945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:40.359165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 858/1722] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:40 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:40.360555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:40.385677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402737b1-9113-4cf5-bacf-c4c1d85934c0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:40.392989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:40.394060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:40.394384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:40.394660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:40.395591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:40.399352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:40.421943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2cdb9-0181-4509-8035-a75de687c3d8 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:40.427174 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 859/1723] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:40.428827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:40.443680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc426e-dcad-4e1a-a5fb-8743a08fdb44 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:40.451264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:40.452833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:40.453373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:40.453665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:40.454810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:40.457143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:40.485483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 860/1724] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:40.488514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:40.513521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863fc93-c3a5-4d13-ab76-7e4ee17c336b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:40.527430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208e730-54ca-4b0e-96d2-893713c129e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:40.532361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208e730-54ca-4b0e-96d2-893713c129e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:40.533068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208e730-54ca-4b0e-96d2-893713c129e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:40.534339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208e730-54ca-4b0e-96d2-893713c129e4 admin admin] PATH_INFO: `/v3/projects/76244caf2ae043c49903f7ffc82f8288/users/0e4f5b6b56904fed85c0e93d32f21713/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:40.583183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208e730-54ca-4b0e-96d2-893713c129e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76244caf2ae043c49903f7ffc82f8288, user_id=0e4f5b6b56904fed85c0e93d32f21713, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:40.588988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208e730-54ca-4b0e-96d2-893713c129e4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:40.597834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:40.599294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:40.600126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:40.600883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:40.638670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 861/1725] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:40 2022] PUT /identity/v3/projects/76244caf2ae043c49903f7ffc82f8288/users/0e4f5b6b56904fed85c0e93d32f21713/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:40.640718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd388af-b034-41b7-9ec1-49f9f52052ae None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:40.641031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd388af-b034-41b7-9ec1-49f9f52052ae None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:40.641312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd388af-b034-41b7-9ec1-49f9f52052ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:40.712425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:40.714730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:40.727217 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-dcd388af-b034-41b7-9ec1-49f9f52052ae None None] MFA Rules not processed for user `0e4f5b6b56904fed85c0e93d32f21713`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:40.728556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-dcd388af-b034-41b7-9ec1-49f9f52052ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:40.730025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 865/1726] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:40.943149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:40.997076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74313b-17a1-411e-ba0f-4cc05fc9509e service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:41.031707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 862/1727] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:41.036715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd388af-b034-41b7-9ec1-49f9f52052ae None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:41.220371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd388af-b034-41b7-9ec1-49f9f52052ae None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:41.241399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:41.242148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:41.242980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:41.242980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:41.244129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:41.246578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:41.329855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 863/1728] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:41 2022] POST /identity/v3/projects => generated 365 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:52:41.330916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:41.365061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:41.366070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:41.369071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:41.369380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] PATH_INFO: `/v3/users/5ac66da619424e6f8ce6d1c45afb376b` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:41.373718 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac66da619424e6f8ce6d1c45afb376b)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:41.375905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:41.519910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac66da619424e6f8ce6d1c45afb376b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:41.521284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd05891-9a49-49e6-b81a-48e064be1de7 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:41.533435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:41.534433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:41.534895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:41.535343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:41.536230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:41.540329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:41.568960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac66da619424e6f8ce6d1c45afb376b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:41.571613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac66da619424e6f8ce6d1c45afb376b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:41.573004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac66da619424e6f8ce6d1c45afb376b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:41.581905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 864/1729] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:41 2022] POST /identity/v3/users => generated 289 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:41.588550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:41.594291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-250f0d9e-0fd5-4a71-950d-3530a3e59836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac66da619424e6f8ce6d1c45afb376b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:41.608594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:41.637600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 866/1730] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:40 2022] DELETE /identity/v3/users/5ac66da619424e6f8ce6d1c45afb376b => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:41.640780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131dea6-a956-4fa4-8f30-36b59455b89b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:41.640780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131dea6-a956-4fa4-8f30-36b59455b89b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:41.640780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131dea6-a956-4fa4-8f30-36b59455b89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:41.678518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983476ad-6754-419d-b664-2852c0567b5a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:41.694044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-7131dea6-a956-4fa4-8f30-36b59455b89b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:41.698347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7131dea6-a956-4fa4-8f30-36b59455b89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:41.819092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:41.821780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:41.822426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:41.823177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:41.824551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:41.826789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:41.843287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 865/1731] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:41.849229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:41.873964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a85227-2f2a-4ab9-8ff3-0918dc1b8121 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:41.890549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc84a-9ed3-4978-873d-80bba42d41f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:41.891813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc84a-9ed3-4978-873d-80bba42d41f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:41.893123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc84a-9ed3-4978-873d-80bba42d41f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:41.894167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc84a-9ed3-4978-873d-80bba42d41f9 admin admin] PATH_INFO: `/v3/projects/001f93e2fce5477d850a75e0ff86ea35/users/28ed412154e0476cab6e1918a39d1b4c/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:41.933681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 867/1732] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:41.948918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbc84a-9ed3-4978-873d-80bba42d41f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001f93e2fce5477d850a75e0ff86ea35, user_id=28ed412154e0476cab6e1918a39d1b4c, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:41.962391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbc84a-9ed3-4978-873d-80bba42d41f9 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:42.001341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 866/1733] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:41 2022] PUT /identity/v3/projects/001f93e2fce5477d850a75e0ff86ea35/users/28ed412154e0476cab6e1918a39d1b4c/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:42.007805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131dea6-a956-4fa4-8f30-36b59455b89b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:42.039231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131dea6-a956-4fa4-8f30-36b59455b89b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:42.077392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.077749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.078003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.190885 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] MFA Rules not processed for user `06d696a14671493db2c243300a19cbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:42.192955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:42.274408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:42.276160 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.276513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.277057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.278093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:42.281563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:42.305892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 868/1734] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:42.306876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:42.318200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:42.368560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 867/1735] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:42.368560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:42.368560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:42.411302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363f1a4-dc5b-4325-a13c-cfdaac96c1cd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:42.426642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:42.427553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.427843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.428118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] PATH_INFO: `/v3/projects/8b09c1a142e649da8492e92e17ac227d` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.430800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b09c1a142e649da8492e92e17ac227d)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:42.433789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:42.467843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97fb6b-5b0d-43ec-9ce2-4fa98e3f2cf6 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:42.478805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:42.486184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.486896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.487264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.540896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:42.543912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:42.558211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b09c1a142e649da8492e92e17ac227d', 'request_id': 'req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:42.571605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 869/1736] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:42.572371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b09c1a142e649da8492e92e17ac227d', 'request_id': 'req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:42.583485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:42.594036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b09c1a142e649da8492e92e17ac227d', 'request_id': 'req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:42.619804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7ab2b-9e75-46b8-b8a1-76b3fe7c71c4 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:42.623023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] The token cache is being invalidate because project 8b09c1a142e649da8492e92e17ac227d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:42.623619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:42.625749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3d2d3d0c-9886-4860-9c29-dbb701d2fb39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:42.629004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 868/1737] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:42 2022] DELETE /identity/v3/projects/8b09c1a142e649da8492e92e17ac227d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:42.634852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526bde-8f7f-49e5-bcea-8f25dfdf6bb8 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.635142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526bde-8f7f-49e5-bcea-8f25dfdf6bb8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.636333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4526bde-8f7f-49e5-bcea-8f25dfdf6bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.689891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c4526bde-8f7f-49e5-bcea-8f25dfdf6bb8 None None] MFA Rules not processed for user `06d696a14671493db2c243300a19cbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:42.697820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c4526bde-8f7f-49e5-bcea-8f25dfdf6bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:42.805772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f686764-be0c-4a8c-9cd4-56a16fb69d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:42.807390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f686764-be0c-4a8c-9cd4-56a16fb69d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.811611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f686764-be0c-4a8c-9cd4-56a16fb69d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.812187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f686764-be0c-4a8c-9cd4-56a16fb69d33 admin admin] PATH_INFO: `/v3/projects/001f93e2fce5477d850a75e0ff86ea35/users/28ed412154e0476cab6e1918a39d1b4c/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.815152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f686764-be0c-4a8c-9cd4-56a16fb69d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001f93e2fce5477d850a75e0ff86ea35, user_id=28ed412154e0476cab6e1918a39d1b4c, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:42.821081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f686764-be0c-4a8c-9cd4-56a16fb69d33 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:42.839409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 870/1738] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:42 2022] PUT /identity/v3/projects/001f93e2fce5477d850a75e0ff86ea35/users/28ed412154e0476cab6e1918a39d1b4c/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:42.841316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.841654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.841963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.872897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 869/1739] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:42.872897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:42.872897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:42.872897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:42.879406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:42.882760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:42.976555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] MFA Rules not processed for user `28ed412154e0476cab6e1918a39d1b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:42.977733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.183143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 870/1740] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:43.183143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:43.193738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.270416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 871/1741] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:43.272115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:43.280220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.372510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648bb57-7fde-42cb-9a48-1f29cabf161c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:43.391384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:43.392176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:43.392467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:43.392739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:43.402412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.466962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67e7c3-d92a-468c-8593-39bdbb43badb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:43.478640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25f17b-9fb4-4204-a424-ee4975ad07ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:43.479325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25f17b-9fb4-4204-a424-ee4975ad07ad service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:43.479604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25f17b-9fb4-4204-a424-ee4975ad07ad service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:43.479874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25f17b-9fb4-4204-a424-ee4975ad07ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:43.498343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25f17b-9fb4-4204-a424-ee4975ad07ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:43.500376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25f17b-9fb4-4204-a424-ee4975ad07ad service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:43.527313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 872/1742] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:43.528431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13802-5d26-495e-a589-38881262dc9b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:43.528713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13802-5d26-495e-a589-38881262dc9b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:43.529034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da13802-5d26-495e-a589-38881262dc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:43.592892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:43.594654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:43.610468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1da13802-5d26-495e-a589-38881262dc9b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:43.611713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1da13802-5d26-495e-a589-38881262dc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.634643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 871/1743] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:43.651150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:43.660163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.663826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 873/1744] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:43.682048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13802-5d26-495e-a589-38881262dc9b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:43.713877 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da13802-5d26-495e-a589-38881262dc9b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:43.721184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:43.721917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:43.722196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:43.723091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:43.723820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:43.724130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:43.725983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:43.731606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421d61-dcc6-4da5-8fe0-902217347a69 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:43.740451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 874/1745] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:43.741481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:43.749619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:43.752363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:43.754712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:43.755943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] PATH_INFO: `/v3/projects/a8657b6e1ba1460987724472f11a00e4` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:43.761346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8657b6e1ba1460987724472f11a00e4)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:43.767690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d832351-e2a6-4119-821e-204326259160 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:43.769242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:43.776553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:43.777490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:43.777867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:43.778235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:43.782762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.877339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8657b6e1ba1460987724472f11a00e4', 'request_id': 'req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:43.885669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8657b6e1ba1460987724472f11a00e4', 'request_id': 'req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:43.926463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8657b6e1ba1460987724472f11a00e4', 'request_id': 'req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:43.964197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:43.967785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] The token cache is being invalidate because project a8657b6e1ba1460987724472f11a00e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:43.968598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:43.968783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:43.970012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:43.973348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 872/1746] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:43 2022] DELETE /identity/v3/projects/a8657b6e1ba1460987724472f11a00e4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:43.974974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:43.977701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:43.994185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 875/1747] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:43.996320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:44.027196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f4e87-ef18-4495-bf86-236cd37c83eb service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:44.175939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d617a-f3b2-49c0-9907-6b5bbafd53b7 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:44.198247 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:44.199354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:44.199641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:44.203057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:44.203980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:44.212427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:44.218530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:44.222254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:44.222588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:44.222876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] PATH_INFO: `/v3/users/ac8354829c934761a5f4128855130937` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:44.227488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8354829c934761a5f4128855130937)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:44.232922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:44.286461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 873/1748] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:43 2022] POST /identity/v3/projects => generated 381 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:44.287729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:44.347748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8354829c934761a5f4128855130937'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:44.366661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8354829c934761a5f4128855130937'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:44.367834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8354829c934761a5f4128855130937'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:44.372050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8354829c934761a5f4128855130937'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:44.399698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b82b47a2-3012-449b-a5cc-dd0e27d941b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8354829c934761a5f4128855130937'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:44.451787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 876/1749] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:43 2022] DELETE /identity/v3/users/ac8354829c934761a5f4128855130937 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:44.530950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2bc82-067d-4721-951f-9c5d624b7d17 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:44.577385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:44.578564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:44.579607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:44.654076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] MFA Rules not processed for user `da47bf4c34f243bb9493d8bc11891cef`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:44.655359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:44.728972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:44.730203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:44.730540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:44.730846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:44.731803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:44.734901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:44.777568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 877/1750] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:44.788865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 874/1751] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:44 2022] POST /identity/v3/users => generated 291 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:44.790785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:44.798167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:44.817570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815bd2b-926a-408b-8857-3a77824ea103 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:44.832044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259ef34-4106-44c1-bf66-aa234704b334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:44.833730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ef34-4106-44c1-bf66-aa234704b334 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:44.834072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ef34-4106-44c1-bf66-aa234704b334 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:44.834455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259ef34-4106-44c1-bf66-aa234704b334 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:44.835481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259ef34-4106-44c1-bf66-aa234704b334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:44.839253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259ef34-4106-44c1-bf66-aa234704b334 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:44.871246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 878/1752] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:44.872356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:44.886495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779edb76-a156-4248-b259-3da8fc477d6a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:44.899493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1d43b-1526-416a-8cbd-562315813abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:44.900265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1d43b-1526-416a-8cbd-562315813abc admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:44.900551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1d43b-1526-416a-8cbd-562315813abc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:44.900850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1d43b-1526-416a-8cbd-562315813abc admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:44.901636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1d43b-1526-416a-8cbd-562315813abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:44.903694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1d43b-1526-416a-8cbd-562315813abc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:44.965051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf83407-498d-4ee1-8163-d974f332091c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:44.966034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf83407-498d-4ee1-8163-d974f332091c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:44.966493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf83407-498d-4ee1-8163-d974f332091c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:44.969561 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 879/1753] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 4 14:52:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:44.979329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 875/1754] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:44.985666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf83407-498d-4ee1-8163-d974f332091c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:45.002748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:45.003138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:45.003467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:45.018767 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf83407-498d-4ee1-8163-d974f332091c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:45.037115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da9d93-3d8e-4320-8b4d-0cd81583fd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:45.038052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da9d93-3d8e-4320-8b4d-0cd81583fd32 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:45.038331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da9d93-3d8e-4320-8b4d-0cd81583fd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:45.038604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da9d93-3d8e-4320-8b4d-0cd81583fd32 admin admin] PATH_INFO: `/v3/projects/99b786b3acb946b794388d710cd6340f/users/d706a86d6ba24e428d47257ffe523c5b/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:45.050351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] MFA Rules not processed for user `0d678d5ee7d74896bbc1316b32ec81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:45.052748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:45.082461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da9d93-3d8e-4320-8b4d-0cd81583fd32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b786b3acb946b794388d710cd6340f, user_id=d706a86d6ba24e428d47257ffe523c5b, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:45.089736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da9d93-3d8e-4320-8b4d-0cd81583fd32 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:45.118857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 880/1755] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:44 2022] PUT /identity/v3/projects/99b786b3acb946b794388d710cd6340f/users/d706a86d6ba24e428d47257ffe523c5b/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:45.136848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca42b-9c1b-497b-a844-8130aacfbd3a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:45.137186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca42b-9c1b-497b-a844-8130aacfbd3a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:45.137401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dca42b-9c1b-497b-a844-8130aacfbd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:45.171200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 876/1756] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 04 14:52:45.196073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-95dca42b-9c1b-497b-a844-8130aacfbd3a None None] MFA Rules not processed for user `d706a86d6ba24e428d47257ffe523c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:45.198753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-95dca42b-9c1b-497b-a844-8130aacfbd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:45.223154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:45.224596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:45.240939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d32bf-4a36-4bc4-9420-3bf0160e9cf2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:45.365525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:45.366388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:45.366677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:45.367016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:45.380479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:45.382867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:45.409947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 877/1757] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:45.451198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 881/1758] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:45.457493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:45.678226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c881876-51aa-4c7f-b78d-75cb6e2572b9 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:45.692152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:45.693056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:45.693353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:45.694007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:45.695674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:45.698198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:45.737519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 878/1759] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:45 2022] POST /identity/v3/projects => generated 381 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:45.741353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dca42b-9c1b-497b-a844-8130aacfbd3a None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:45.760568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dca42b-9c1b-497b-a844-8130aacfbd3a None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:45.856855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:45.862214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:45.926170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a726d5-0677-410f-951f-3997076b78a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:45.926980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a726d5-0677-410f-951f-3997076b78a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:45.927320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a726d5-0677-410f-951f-3997076b78a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:45.927595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a726d5-0677-410f-951f-3997076b78a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:45.928328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a726d5-0677-410f-951f-3997076b78a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:45.935011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a726d5-0677-410f-951f-3997076b78a5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:46.040309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ac1b-3d13-4a84-b77e-53c941c20b68 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:46.068471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:46.069709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.070384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.070873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] PATH_INFO: `/v3/projects/ae7406b07f984d8bb1f70d5fb7022f8f` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.089357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7406b07f984d8bb1f70d5fb7022f8f)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:46.093059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:46.106599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 882/1760] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:45 2022] POST /identity/v3/users => generated 297 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:52:46.116013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627718e-b3d0-4f1a-b311-e05d473bea3d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.122730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627718e-b3d0-4f1a-b311-e05d473bea3d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.130742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627718e-b3d0-4f1a-b311-e05d473bea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.260914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-9627718e-b3d0-4f1a-b311-e05d473bea3d None None] MFA Rules not processed for user `aff6489e5a6441509376c0dca551f03c`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:46.263706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9627718e-b3d0-4f1a-b311-e05d473bea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:46.267263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7406b07f984d8bb1f70d5fb7022f8f', 'request_id': 'req-5d559c6b-35bf-42ba-adbe-7853bfcc68df'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:46.281975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7406b07f984d8bb1f70d5fb7022f8f', 'request_id': 'req-5d559c6b-35bf-42ba-adbe-7853bfcc68df'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:46.299315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7406b07f984d8bb1f70d5fb7022f8f', 'request_id': 'req-5d559c6b-35bf-42ba-adbe-7853bfcc68df'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:46.343801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] The token cache is being invalidate because project ae7406b07f984d8bb1f70d5fb7022f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:46.346520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:46.348661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:46.360050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 879/1761] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:45 2022] DELETE /identity/v3/projects/ae7406b07f984d8bb1f70d5fb7022f8f => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:52:46.360050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:46.364916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:46.535668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 883/1762] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:46.537973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d559c6b-35bf-42ba-adbe-7853bfcc68df admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:46.539342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef463d-6576-4fdf-b996-338d48109ab3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.539660 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef463d-6576-4fdf-b996-338d48109ab3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.539941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef463d-6576-4fdf-b996-338d48109ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.559357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:46.560066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.560369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.560691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.561469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:46.578774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:46.613182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 880/1763] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:46.613507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-5aef463d-6576-4fdf-b996-338d48109ab3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:46.613598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:46.616118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5aef463d-6576-4fdf-b996-338d48109ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:46.636609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 884/1764] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:46.640302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1073d-971e-4dc5-8597-314e5e3ca458 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.640612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1073d-971e-4dc5-8597-314e5e3ca458 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.641072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1073d-971e-4dc5-8597-314e5e3ca458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.656032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686cd2a-7f10-4309-8993-4966e23dbe89 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:46.685070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:46.686903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.687219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.687503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] PATH_INFO: `/v3/projects/7f072c7be0a648b9a3efa6d0d9c7e5f1/users/56e99c1c9eac4aee873c977a41e7a221/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.716744 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d3a1073d-971e-4dc5-8597-314e5e3ca458 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:46.719244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f072c7be0a648b9a3efa6d0d9c7e5f1, user_id=56e99c1c9eac4aee873c977a41e7a221, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:46.720726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d3a1073d-971e-4dc5-8597-314e5e3ca458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:46.722479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:46.747414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 885/1765] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:46.753230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1073d-971e-4dc5-8597-314e5e3ca458 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:46.775403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 881/1766] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:46 2022] PUT /identity/v3/projects/7f072c7be0a648b9a3efa6d0d9c7e5f1/users/56e99c1c9eac4aee873c977a41e7a221/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:46.781574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:46.781878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1073d-971e-4dc5-8597-314e5e3ca458 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:46.886324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:46.887061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.887732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.887732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.888490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:46.888702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:46.890900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:46.917970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 886/1767] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:46.929237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:46.952081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b57ce-4622-46d0-9eda-4e6b26d67b40 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:46.965223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a82f7-99f4-47cf-bac0-431f1bf99a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:46.969375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a82f7-99f4-47cf-bac0-431f1bf99a12 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:46.969935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a82f7-99f4-47cf-bac0-431f1bf99a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:46.970525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a82f7-99f4-47cf-bac0-431f1bf99a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:46.971439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a82f7-99f4-47cf-bac0-431f1bf99a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:46.974991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a82f7-99f4-47cf-bac0-431f1bf99a12 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:46.982225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bb15e-3cd7-45ec-a00c-990c49e63ec4 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:47.002502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:47.003507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:47.003965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:47.005448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:47.007161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:47.008064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:47.048431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 887/1768] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:46 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:47.049908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9062d-0a63-485c-bd7b-ed5a6b0faf98 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:47.050359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9062d-0a63-485c-bd7b-ed5a6b0faf98 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:47.050775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9062d-0a63-485c-bd7b-ed5a6b0faf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:47.059018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 882/1769] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:47.062806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:47.101655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c2b9062d-0a63-485c-bd7b-ed5a6b0faf98 None None] MFA Rules not processed for user `06d696a14671493db2c243300a19cbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:47.103114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6344467-9a01-46fe-805f-682fc06058ca admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:47.104986 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c2b9062d-0a63-485c-bd7b-ed5a6b0faf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:47.212472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 888/1770] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:47.220827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9062d-0a63-485c-bd7b-ed5a6b0faf98 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:47.277919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:47.281280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:47.283388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:47.284072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:47.284835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:47.287040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:47.344464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 883/1771] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:47 2022] POST /identity/v3/users => generated 291 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:47.345856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:47.354379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:47.397037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9062d-0a63-485c-bd7b-ed5a6b0faf98 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:47.405051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:47.406583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:47.407055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:47.407770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] PATH_INFO: `/v3/projects/7f072c7be0a648b9a3efa6d0d9c7e5f1/users/56e99c1c9eac4aee873c977a41e7a221/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:47.414021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f072c7be0a648b9a3efa6d0d9c7e5f1, user_id=56e99c1c9eac4aee873c977a41e7a221, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:47.420582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:47.441271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 889/1772] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:47 2022] PUT /identity/v3/projects/7f072c7be0a648b9a3efa6d0d9c7e5f1/users/56e99c1c9eac4aee873c977a41e7a221/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:47.443031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:47.632704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831cfa5-8d6c-40b5-ba95-8f6ca09b6ee2 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:47.680739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb7622-5b87-4690-9a43-5facf77c7716 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:47.704874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68fd3-7c30-4e4b-9f0e-ac600346d35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:47.705609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68fd3-7c30-4e4b-9f0e-ac600346d35b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:47.705912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68fd3-7c30-4e4b-9f0e-ac600346d35b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:47.708265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68fd3-7c30-4e4b-9f0e-ac600346d35b admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:47.710502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68fd3-7c30-4e4b-9f0e-ac600346d35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:47.715711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68fd3-7c30-4e4b-9f0e-ac600346d35b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:47.766262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 890/1773] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:47.771785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35e180-c5fe-404f-a4db-5403c2082c23 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:47.775584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35e180-c5fe-404f-a4db-5403c2082c23 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:47.777405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35e180-c5fe-404f-a4db-5403c2082c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:47.870029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2a35e180-c5fe-404f-a4db-5403c2082c23 None None] MFA Rules not processed for user `56e99c1c9eac4aee873c977a41e7a221`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:47.872244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2a35e180-c5fe-404f-a4db-5403c2082c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:47.950904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:47.955416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:47.955886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:47.956360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:48.132334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 891/1774] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:48.133558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35e180-c5fe-404f-a4db-5403c2082c23 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:48.168061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35e180-c5fe-404f-a4db-5403c2082c23 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:48.179542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:48.180247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:48.181534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:48.181534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:48.181742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:48.184252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:48.210562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 892/1775] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:48.224955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:48.228725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:48.270874 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:48.276617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:48.299188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 884/1776] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 954 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:48.300279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:48.329098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae81056-6a49-4c76-b64a-ddc0a9d5fd50 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:48.344453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1d39d-afa9-4d5d-a834-41ef4017fa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:48.348264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d39d-afa9-4d5d-a834-41ef4017fa46 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:48.352697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d39d-afa9-4d5d-a834-41ef4017fa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:48.357722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d39d-afa9-4d5d-a834-41ef4017fa46 admin admin] PATH_INFO: `/v3/projects/6d7e0a37c21a4c3b94cde1a16cfd79a7/users/35f9872284ac445d9c922371066073ad/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:48.431614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1d39d-afa9-4d5d-a834-41ef4017fa46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7e0a37c21a4c3b94cde1a16cfd79a7, user_id=35f9872284ac445d9c922371066073ad, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:48.443154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1d39d-afa9-4d5d-a834-41ef4017fa46 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:48.476066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 885/1777] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:48 2022] PUT /identity/v3/projects/6d7e0a37c21a4c3b94cde1a16cfd79a7/users/35f9872284ac445d9c922371066073ad/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:48.494074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7bf5b-2de7-4303-9d28-7aa83f9f434a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:48.494528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7bf5b-2de7-4303-9d28-7aa83f9f434a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:48.495161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7bf5b-2de7-4303-9d28-7aa83f9f434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:48.557046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8801659-594b-4f0a-82d1-12eb0ac7fcd1 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:48.572128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0dc7bf5b-2de7-4303-9d28-7aa83f9f434a None None] MFA Rules not processed for user `35f9872284ac445d9c922371066073ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:48.578158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0dc7bf5b-2de7-4303-9d28-7aa83f9f434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:48.754414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 886/1778] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:48.761457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7bf5b-2de7-4303-9d28-7aa83f9f434a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:48.819090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:48.820039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:48.820443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:48.820871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:49.138701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7bf5b-2de7-4303-9d28-7aa83f9f434a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:49.166846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:49.171947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:49.173322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:49.177828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:49.186639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:49.188350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:49.191684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:49.207678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:49.289477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 893/1779] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 1078 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:49.345999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 887/1780] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:48 2022] POST /identity/v3/projects => generated 381 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:49.353839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:49.388247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862bed9-a097-4b8c-a71b-c4a5ddd7672d service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:49.662642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:49.663802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:49.664273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:49.664748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:49.668783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:49.674318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:49.731393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 894/1781] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:52:49 2022] POST /identity/v3/users => generated 297 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:52:49.735753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:49.765527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d105dd-d832-42a2-974f-81784675ae95 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:49.788404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:49.789480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:49.789909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:49.790333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:49.791242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:49.793065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:49.821103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 888/1782] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:49.826831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:49.843482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415efe-77ac-4c55-9828-27a03311eea4 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:49.860294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:49.862109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:49.862699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:49.863308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] PATH_INFO: `/v3/projects/944b6f49a35a49b8a83ae8b098c00296/users/3977c7307ea24564bf360ea5469b457e/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:49.901163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=944b6f49a35a49b8a83ae8b098c00296, user_id=3977c7307ea24564bf360ea5469b457e, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:49.904002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:49.937965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 895/1783] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:49 2022] PUT /identity/v3/projects/944b6f49a35a49b8a83ae8b098c00296/users/3977c7307ea24564bf360ea5469b457e/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:49.939237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:49.961676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444f6e-2262-4963-bd2e-5db301ef59dc admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:50.117250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:50.118077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:50.118461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:50.118843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:50.119849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:50.124983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:50.158041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 889/1784] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:52:50.159421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:50.173999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:50.176525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3826e7d7-d76a-4b3f-966d-da85a00b07dd admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:50.189669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8df17-02d4-4911-b32c-19109d62ca6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:50.190646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8df17-02d4-4911-b32c-19109d62ca6b admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:50.190974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8df17-02d4-4911-b32c-19109d62ca6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:50.191434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8df17-02d4-4911-b32c-19109d62ca6b admin admin] PATH_INFO: `/v3/projects/944b6f49a35a49b8a83ae8b098c00296/users/3977c7307ea24564bf360ea5469b457e/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:50.201948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8df17-02d4-4911-b32c-19109d62ca6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=944b6f49a35a49b8a83ae8b098c00296, user_id=3977c7307ea24564bf360ea5469b457e, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:50.205758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8df17-02d4-4911-b32c-19109d62ca6b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:50.235448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 896/1785] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:52:50 2022] PUT /identity/v3/projects/944b6f49a35a49b8a83ae8b098c00296/users/3977c7307ea24564bf360ea5469b457e/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:50.260042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13456-f892-4684-9826-efdabb796cbe None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:50.260687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13456-f892-4684-9826-efdabb796cbe None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:50.261140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a13456-f892-4684-9826-efdabb796cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:50.316962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-44a13456-f892-4684-9826-efdabb796cbe None None] MFA Rules not processed for user `3977c7307ea24564bf360ea5469b457e`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:50.318436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-44a13456-f892-4684-9826-efdabb796cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:50.441241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984d27c-9541-417c-986b-ab1afc3d89a5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:50.532922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 897/1786] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:50.550125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a13456-f892-4684-9826-efdabb796cbe None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:50.633672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e99c1c9eac4aee873c977a41e7a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f072c7be0a648b9a3efa6d0d9c7e5f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:50.634412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:50.635277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:50.635394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:50.637116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:50.637463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:50.640276 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:50.668399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 890/1787] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:52:50.675516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:50.696229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7f670-e6ed-4119-8cf2-96f935f62aea tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:50.711364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b4cd8-f3a7-4682-9a5b-2b04904e9f77 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e99c1c9eac4aee873c977a41e7a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f072c7be0a648b9a3efa6d0d9c7e5f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:50.713387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b4cd8-f3a7-4682-9a5b-2b04904e9f77 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:50.714454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b4cd8-f3a7-4682-9a5b-2b04904e9f77 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:50.714929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b4cd8-f3a7-4682-9a5b-2b04904e9f77 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:50.717205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b4cd8-f3a7-4682-9a5b-2b04904e9f77 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:50.721600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b4cd8-f3a7-4682-9a5b-2b04904e9f77 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:50.783591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a13456-f892-4684-9826-efdabb796cbe None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:50.800678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 891/1788] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:52:50 2022] POST /identity/v3/projects => generated 354 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:52:50.802329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:50.802615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:50.802934 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:50.808616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:50.810320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:50.811402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:50.813160 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:50.862604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:50.869494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:50.890126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] MFA Rules not processed for user `a0944ce1e8504fb1b50507ace4e63649`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:50.891487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:50.918716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 898/1789] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:50.921426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:50.944191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a0231-fc28-4df5-b0b8-e4de37d2c766 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:51.190552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 892/1790] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:51.246670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e99c1c9eac4aee873c977a41e7a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f072c7be0a648b9a3efa6d0d9c7e5f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:51.248696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:51.249773 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:51.250627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:51.252150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:52:51.252825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:51.256231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:51.270766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:51.273117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:51.295770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 899/1791] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:52:51.408669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:51.413607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:51.467553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27a0ab-a6d7-4c96-9495-91109a347d22 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:51.506668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:51.512191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:51.515319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:51.517288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:51.702595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:51.704377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:51.730128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 893/1792] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:51.741650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:51.758620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc790839-c995-481e-a164-0591d1eb38ba service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:51.796924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56e99c1c9eac4aee873c977a41e7a221', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f072c7be0a648b9a3efa6d0d9c7e5f1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:51.803433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:51.803736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:51.804031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] PATH_INFO: `/v3/projects/8bb4a98a92c0478cb2c9ac97ab533213` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:51.809763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8bb4a98a92c0478cb2c9ac97ab533213)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:51.811943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:51.833131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b9fa1-9f14-4234-846c-89d6bde6aecd tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:51.862686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:51.865945 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:51.872888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:51.874681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:51.926232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:51.930064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:51.955640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 900/1793] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:52.074997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb4a98a92c0478cb2c9ac97ab533213', 'request_id': 'req-55daecc4-0066-48d7-8a75-1dea90c6be38'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:52.106267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb4a98a92c0478cb2c9ac97ab533213', 'request_id': 'req-55daecc4-0066-48d7-8a75-1dea90c6be38'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:52.130189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb4a98a92c0478cb2c9ac97ab533213', 'request_id': 'req-55daecc4-0066-48d7-8a75-1dea90c6be38'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:52.159277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] The token cache is being invalidate because project 8bb4a98a92c0478cb2c9ac97ab533213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:52.159889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:52.161465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-55daecc4-0066-48d7-8a75-1dea90c6be38 tempest-VolumeQuotaClassesTest-1599182893 tempest-VolumeQuotaClassesTest-1599182893-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:52.166924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 894/1794] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:51 2022] DELETE /identity/v3/projects/8bb4a98a92c0478cb2c9ac97ab533213 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:52.882641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:53.056646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d47f66-93c8-449c-abef-6597b93b16f1 service nova] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:53.230056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:53.231146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:53.231721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:53.232170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] PATH_INFO: `/v3/users/d706a86d6ba24e428d47257ffe523c5b` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:53.237243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d706a86d6ba24e428d47257ffe523c5b)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:53.240235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:53.454564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd706a86d6ba24e428d47257ffe523c5b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:53.554507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd706a86d6ba24e428d47257ffe523c5b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:53.559220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd706a86d6ba24e428d47257ffe523c5b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:53.562437 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd706a86d6ba24e428d47257ffe523c5b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:53.647832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd706a86d6ba24e428d47257ffe523c5b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:53.701348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 901/1795] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:52 2022] DELETE /identity/v3/users/d706a86d6ba24e428d47257ffe523c5b => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:53.707535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022f395-bd9d-4d3f-b38b-eeed38151152 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:53.708346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022f395-bd9d-4d3f-b38b-eeed38151152 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:53.708706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022f395-bd9d-4d3f-b38b-eeed38151152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:53.770537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b022f395-bd9d-4d3f-b38b-eeed38151152 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:53.772329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b022f395-bd9d-4d3f-b38b-eeed38151152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:53.915894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 895/1796] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:53.951621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:53.953851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:54.128628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88d1d4-610a-4280-85a7-992bb94d01a0 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:54.134014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:54.135549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:54.136164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:54.137586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:54.160459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:54.162391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:54.187287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 902/1797] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:55.883711 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac89cc-6b0a-466b-a5b9-f8ed5737a044 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:55.884364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac89cc-6b0a-466b-a5b9-f8ed5737a044 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:55.884697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac89cc-6b0a-466b-a5b9-f8ed5737a044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:55.935734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0fac89cc-6b0a-466b-a5b9-f8ed5737a044 None None] MFA Rules not processed for user `aff6489e5a6441509376c0dca551f03c`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:55.944029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0fac89cc-6b0a-466b-a5b9-f8ed5737a044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:56.061480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:56.064475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:56.131929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7a1e0-b067-4ebd-92c8-9004f9a7be24 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:56.136052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 896/1798] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:56.143790 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:56.145196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:56.145828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:56.146375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] PATH_INFO: `/v3/users/35f9872284ac445d9c922371066073ad` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:56.149068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f9872284ac445d9c922371066073ad)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:56.151448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:56.285096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f9872284ac445d9c922371066073ad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:56.327605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f9872284ac445d9c922371066073ad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:56.329434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f9872284ac445d9c922371066073ad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:56.333071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f9872284ac445d9c922371066073ad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:56.351887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f9872284ac445d9c922371066073ad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:56.403086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 903/1799] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:56 2022] DELETE /identity/v3/users/35f9872284ac445d9c922371066073ad => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:56.408358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:56.408923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:56.409379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:56.463383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:56.472164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:56.609231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:56.611834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:56.687547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 897/1800] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:56.747190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:56.749093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:56.801504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b15e8-6ead-47f9-a047-a9d2395cf1ed admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:56.854427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:56.859980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:56.863521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:56.863959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] PATH_INFO: `/v3/projects/99b786b3acb946b794388d710cd6340f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:56.870490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b786b3acb946b794388d710cd6340f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:56.877327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:56.926191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88354245-9cdf-4f4f-a24d-317c21e07287 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:56.954864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:56.958063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:56.958351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:56.958776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:56.990726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:57.000784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:57.036621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 898/1801] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:57.087478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b786b3acb946b794388d710cd6340f', 'request_id': 'req-94361278-b808-4e1c-bce6-9fd5aad0ba6c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.096889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b786b3acb946b794388d710cd6340f', 'request_id': 'req-94361278-b808-4e1c-bce6-9fd5aad0ba6c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.105530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b786b3acb946b794388d710cd6340f', 'request_id': 'req-94361278-b808-4e1c-bce6-9fd5aad0ba6c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.123140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] The token cache is being invalidate because project 99b786b3acb946b794388d710cd6340f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:57.123686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.124883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94361278-b808-4e1c-bce6-9fd5aad0ba6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.132426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 904/1802] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:56 2022] DELETE /identity/v3/projects/99b786b3acb946b794388d710cd6340f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:57.136628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:57.151113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ebe06-6e27-4480-b1c3-f548e8081ba3 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:57.287864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:57.289034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:57.289668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:57.290095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] PATH_INFO: `/v3/users/56e99c1c9eac4aee873c977a41e7a221` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:57.291830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e99c1c9eac4aee873c977a41e7a221)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:57.294080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:57.393540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e99c1c9eac4aee873c977a41e7a221'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.395143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e99c1c9eac4aee873c977a41e7a221'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.396620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e99c1c9eac4aee873c977a41e7a221'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.409867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d10d2-8511-49cf-9db0-ab1be575c306 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:57.410436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d10d2-8511-49cf-9db0-ab1be575c306 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:57.410862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d10d2-8511-49cf-9db0-ab1be575c306 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:57.413091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 905/1803] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:52:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:52:57.417914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e99c1c9eac4aee873c977a41e7a221'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.443639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a8c17-bb6e-47c8-95bc-6f1f386a1395 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:57.444097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a8c17-bb6e-47c8-95bc-6f1f386a1395 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:57.444475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a8c17-bb6e-47c8-95bc-6f1f386a1395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:57.457014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e99c1c9eac4aee873c977a41e7a221'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:57.480120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-338a8c17-bb6e-47c8-95bc-6f1f386a1395 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:57.483578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-338a8c17-bb6e-47c8-95bc-6f1f386a1395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:57.538800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 899/1804] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:57 2022] DELETE /identity/v3/users/56e99c1c9eac4aee873c977a41e7a221 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:57.709735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 906/1805] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:52:57.748266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:57.754721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:57.930359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158793a-1735-4562-813a-4a9b36ec0422 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:57.941664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:57.942670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:57.942964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:57.943246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] PATH_INFO: `/v3/users/0e4f5b6b56904fed85c0e93d32f21713` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:57.944909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4f5b6b56904fed85c0e93d32f21713)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:57.952059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:58.067301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4f5b6b56904fed85c0e93d32f21713'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:58.069893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4f5b6b56904fed85c0e93d32f21713'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:58.071466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4f5b6b56904fed85c0e93d32f21713'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:58.088181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4f5b6b56904fed85c0e93d32f21713'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:58.118794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4f5b6b56904fed85c0e93d32f21713'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:58.145184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 900/1806] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:57 2022] DELETE /identity/v3/users/0e4f5b6b56904fed85c0e93d32f21713 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:58.151274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:58.151918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:58.152071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:58.184778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:52:58.187365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:58.247840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:58.249740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:58.326198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 907/1807] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:52:58.349709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:58.359319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:58.428366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782bad0-c927-44a6-bf5f-d74b26a031ca admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:58.449886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:58.450575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:58.450862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:58.451138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:58.454944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:58.516934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0813d51-05c1-46a3-9f93-092e9ed99ae1 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:58.531861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:58.533350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:58.533529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:58.534891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:58.560924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:58.567077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:58.570538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:58.571766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:58.603338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 908/1808] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:52:58.604887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 901/1809] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:52:59.347361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:59.351059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:52:59.399793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe59670-a48e-4cbd-996f-806f95f596fb service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:59.413813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:59.415841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:59.416684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:59.417573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] PATH_INFO: `/v3/projects/6d7e0a37c21a4c3b94cde1a16cfd79a7` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:59.420731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d7e0a37c21a4c3b94cde1a16cfd79a7)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:59.425006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:59.535243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d7e0a37c21a4c3b94cde1a16cfd79a7', 'request_id': 'req-b2a73137-22d1-4e01-aba3-ffdf1d713cba'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.548306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d7e0a37c21a4c3b94cde1a16cfd79a7', 'request_id': 'req-b2a73137-22d1-4e01-aba3-ffdf1d713cba'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.567275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d7e0a37c21a4c3b94cde1a16cfd79a7', 'request_id': 'req-b2a73137-22d1-4e01-aba3-ffdf1d713cba'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.611562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] The token cache is being invalidate because project 6d7e0a37c21a4c3b94cde1a16cfd79a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:52:59.612075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.615883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.619558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 909/1810] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:52:59 2022] DELETE /identity/v3/projects/6d7e0a37c21a4c3b94cde1a16cfd79a7 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:52:59.623535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:52:59.641491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99343485-d21d-485c-a6cd-4581228f1e94 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:52:59.786806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:52:59.787609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:52:59.788065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:52:59.788489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] PATH_INFO: `/v3/users/3977c7307ea24564bf360ea5469b457e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:52:59.790117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3977c7307ea24564bf360ea5469b457e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:52:59.792702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:52:59.913379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3977c7307ea24564bf360ea5469b457e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.921501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3977c7307ea24564bf360ea5469b457e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.923395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3977c7307ea24564bf360ea5469b457e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.940338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3977c7307ea24564bf360ea5469b457e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:52:59.976498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3977c7307ea24564bf360ea5469b457e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:00.001152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 902/1811] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:52:59 2022] DELETE /identity/v3/users/3977c7307ea24564bf360ea5469b457e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:01.127168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:01.128682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:01.151729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73137-22d1-4e01-aba3-ffdf1d713cba admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:01.415590 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:01.420093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:01.420416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:01.420710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] PATH_INFO: `/v3/projects/7f072c7be0a648b9a3efa6d0d9c7e5f1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:01.438229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f072c7be0a648b9a3efa6d0d9c7e5f1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:01.441745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:01.584577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f072c7be0a648b9a3efa6d0d9c7e5f1', 'request_id': 'req-b2306288-6507-4da2-a8fb-45924fd42c77'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:01.595292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:01.598304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:01.602131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f072c7be0a648b9a3efa6d0d9c7e5f1', 'request_id': 'req-b2306288-6507-4da2-a8fb-45924fd42c77'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:01.635679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8819fe7-8999-4fe0-b8b3-2ebc3c319009 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:01.640885 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f072c7be0a648b9a3efa6d0d9c7e5f1', 'request_id': 'req-b2306288-6507-4da2-a8fb-45924fd42c77'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:01.665113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] The token cache is being invalidate because project 7f072c7be0a648b9a3efa6d0d9c7e5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:01.665113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:01.665113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2306288-6507-4da2-a8fb-45924fd42c77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:01.671969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 910/1812] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:01 2022] DELETE /identity/v3/projects/7f072c7be0a648b9a3efa6d0d9c7e5f1 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:01.674372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de35d5a-130a-4f08-9e35-300622e2ef71 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:01.674735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de35d5a-130a-4f08-9e35-300622e2ef71 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:01.675016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de35d5a-130a-4f08-9e35-300622e2ef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:01.701311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-3de35d5a-130a-4f08-9e35-300622e2ef71 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:01.702899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-3de35d5a-130a-4f08-9e35-300622e2ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:01.811923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 911/1813] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:01.813418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3b0d3-7491-46e1-be3b-8323199f1059 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:01.813706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3b0d3-7491-46e1-be3b-8323199f1059 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:01.814065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3b0d3-7491-46e1-be3b-8323199f1059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:01.828978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:01.829952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:01.830235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:01.830569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] PATH_INFO: `/v3/projects/76244caf2ae043c49903f7ffc82f8288` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:01.837299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=76244caf2ae043c49903f7ffc82f8288)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:01.837535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e7a3b0d3-7491-46e1-be3b-8323199f1059 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:01.838680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e7a3b0d3-7491-46e1-be3b-8323199f1059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:01.842146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:01.850887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 912/1814] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:01.857455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3b0d3-7491-46e1-be3b-8323199f1059 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:01.875926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3b0d3-7491-46e1-be3b-8323199f1059 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:01.886102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:01.887048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:01.887447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:01.887796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:01.892443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:01.892890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:01.894837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:01.915329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 913/1815] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:01.919831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:01.934543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912e90-acf6-4d93-b7f2-ad318ace679d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:01.957827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76244caf2ae043c49903f7ffc82f8288', 'request_id': 'req-b8dd8560-aea3-4977-80ac-9567a93a9fef'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:01.973477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76244caf2ae043c49903f7ffc82f8288', 'request_id': 'req-b8dd8560-aea3-4977-80ac-9567a93a9fef'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:01.988281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76244caf2ae043c49903f7ffc82f8288', 'request_id': 'req-b8dd8560-aea3-4977-80ac-9567a93a9fef'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.024464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] The token cache is being invalidate because project 76244caf2ae043c49903f7ffc82f8288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:02.028240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.030312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.037039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 903/1816] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:01 2022] DELETE /identity/v3/projects/76244caf2ae043c49903f7ffc82f8288 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:02.044332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:02.050629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:02.096989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:02.097690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:02.097971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:02.098246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:02.098969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:02.106564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:02.192909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 914/1817] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:01 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:53:02.194569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:02.196496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:02.199548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd8560-aea3-4977-80ac-9567a93a9fef admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:02.338532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bbb14-996e-4c58-8b49-02bfdfabb300 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:02.347241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:02.348073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:02.348335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:02.348615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:02.349381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:02.350224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:02.351367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:02.351847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:02.352360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] PATH_INFO: `/v3/users/28ed412154e0476cab6e1918a39d1b4c` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:02.352890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:02.377160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=28ed412154e0476cab6e1918a39d1b4c)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:02.382789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:02.389568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 915/1818] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:02 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:02.393810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:02.407361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30813d-39b3-4b0d-9e22-674c5c07d18e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:02.420955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:02.422734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:02.423660 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:02.424032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:02.425010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:02.426807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:02.440513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 916/1819] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:02.444651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:02.463558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbae8f-3b4e-4147-ae93-b2604ce1ebbe admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:02.478156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:02.479129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:02.479518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:02.479897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:02.484607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:02.484607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:02.537558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 917/1820] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:53:02.541626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:02.561322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b3dad-585a-4cc5-a019-7a7c68abb596 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:02.574994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aa787-a8dd-4aa2-897c-28365209dfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:02.575791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa787-a8dd-4aa2-897c-28365209dfb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:02.576178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa787-a8dd-4aa2-897c-28365209dfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:02.576561 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa787-a8dd-4aa2-897c-28365209dfb5 admin admin] PATH_INFO: `/v3/projects/0ae81a31e30e4e338973f430b1cf2c3c/users/a4128309c66e4147a922956161ccfc36/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:02.608969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586aa787-a8dd-4aa2-897c-28365209dfb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae81a31e30e4e338973f430b1cf2c3c, user_id=a4128309c66e4147a922956161ccfc36, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:02.609578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28ed412154e0476cab6e1918a39d1b4c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.611100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28ed412154e0476cab6e1918a39d1b4c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.613725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586aa787-a8dd-4aa2-897c-28365209dfb5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:02.614321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28ed412154e0476cab6e1918a39d1b4c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.631404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28ed412154e0476cab6e1918a39d1b4c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.646913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 918/1821] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:02 2022] PUT /identity/v3/projects/0ae81a31e30e4e338973f430b1cf2c3c/users/a4128309c66e4147a922956161ccfc36/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:53:02.656738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28ed412154e0476cab6e1918a39d1b4c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:02.664478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:02.665661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:02.666120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:02.682436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 904/1822] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:02 2022] DELETE /identity/v3/users/28ed412154e0476cab6e1918a39d1b4c => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:53:02.705951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] MFA Rules not processed for user `a4128309c66e4147a922956161ccfc36`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:02.707582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:02.802185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 919/1823] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:02.808517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:02.810988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:02.951914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:02.953469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:02.985854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1b545-4ffd-47c9-bb39-edda87186cca admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:02.992554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e58a-c737-4940-8e75-2c8b150212e0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:03.004616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:03.005447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.005741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.008296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.009651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:03.010449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:03.011294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.011611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.014019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] PATH_INFO: `/v3/projects/944b6f49a35a49b8a83ae8b098c00296` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.016660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:03.016929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944b6f49a35a49b8a83ae8b098c00296)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:03.023364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:03.088118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 905/1824] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:02 2022] POST /identity/v3/projects => generated 387 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:03.093372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:03.171771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944b6f49a35a49b8a83ae8b098c00296', 'request_id': 'req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.196027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944b6f49a35a49b8a83ae8b098c00296', 'request_id': 'req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.213260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944b6f49a35a49b8a83ae8b098c00296', 'request_id': 'req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.257645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] The token cache is being invalidate because project 944b6f49a35a49b8a83ae8b098c00296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:03.257645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.259233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-76cb74a0-ab19-4b1f-b7d3-ec451fdf8956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.266494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 920/1825] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:02 2022] DELETE /identity/v3/projects/944b6f49a35a49b8a83ae8b098c00296 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:03.268365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7776ec-d94a-4a9c-82d2-3b15c3b3963b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.268692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7776ec-d94a-4a9c-82d2-3b15c3b3963b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.268976 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7776ec-d94a-4a9c-82d2-3b15c3b3963b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.295329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-bf7776ec-d94a-4a9c-82d2-3b15c3b3963b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:03.296719 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bf7776ec-d94a-4a9c-82d2-3b15c3b3963b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:03.311039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6e3c1-489e-4a50-beaa-057caa1de5af admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:03.448570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 921/1826] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:03.458707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a5959-a26c-4a22-b645-5cdcea178602 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.459325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a5959-a26c-4a22-b645-5cdcea178602 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.461091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a5959-a26c-4a22-b645-5cdcea178602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.463336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:03.464068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.464409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.464692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.465642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:03.468414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:03.496556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d65a5959-a26c-4a22-b645-5cdcea178602 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:03.498331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d65a5959-a26c-4a22-b645-5cdcea178602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:03.525379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 922/1827] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:03.526161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a5959-a26c-4a22-b645-5cdcea178602 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:03.536843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 906/1828] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:03 2022] POST /identity/v3/users => generated 294 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:53:03.538130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:03.539198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a5959-a26c-4a22-b645-5cdcea178602 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:03.545549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:03.559518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141d931-d97a-4c68-9b93-a495e1361019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:03.560409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141d931-d97a-4c68-9b93-a495e1361019 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.561126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141d931-d97a-4c68-9b93-a495e1361019 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.562008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141d931-d97a-4c68-9b93-a495e1361019 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.562927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141d931-d97a-4c68-9b93-a495e1361019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:03.563706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141d931-d97a-4c68-9b93-a495e1361019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:03.565979 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141d931-d97a-4c68-9b93-a495e1361019 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:03.576865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee63146-11f5-4cf0-be13-66990b03bfbd admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:03.595163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:03.596141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.596873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.597235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.598080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:03.599957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:03.605097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 923/1829] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:03.608625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855ed9-b8f9-4757-9506-f9cc5c79a7ee None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.609099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855ed9-b8f9-4757-9506-f9cc5c79a7ee None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.609456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30855ed9-b8f9-4757-9506-f9cc5c79a7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.633604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 907/1830] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:03.634913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:03.636625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:03.648483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-30855ed9-b8f9-4757-9506-f9cc5c79a7ee None None] MFA Rules not processed for user `a0944ce1e8504fb1b50507ace4e63649`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:03.650054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-30855ed9-b8f9-4757-9506-f9cc5c79a7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:03.657057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4f5dd-5e66-441e-bad2-b418567e9e26 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:03.662028 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:03.663289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:03.663289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:03.663616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] PATH_INFO: `/v3/projects/001f93e2fce5477d850a75e0ff86ea35` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:03.665059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001f93e2fce5477d850a75e0ff86ea35)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:03.668489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:03.753810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 924/1831] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:03.757552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30855ed9-b8f9-4757-9506-f9cc5c79a7ee None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:03.776337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001f93e2fce5477d850a75e0ff86ea35', 'request_id': 'req-176a144e-1e77-4f20-9508-5d6e59246181'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.795179 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001f93e2fce5477d850a75e0ff86ea35', 'request_id': 'req-176a144e-1e77-4f20-9508-5d6e59246181'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.817995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001f93e2fce5477d850a75e0ff86ea35', 'request_id': 'req-176a144e-1e77-4f20-9508-5d6e59246181'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.833370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30855ed9-b8f9-4757-9506-f9cc5c79a7ee None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:03.865333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] The token cache is being invalidate because project 001f93e2fce5477d850a75e0ff86ea35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:03.865964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.868557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:03.875738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 908/1832] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:03 2022] DELETE /identity/v3/projects/001f93e2fce5477d850a75e0ff86ea35 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:03.893048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:03.898313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:04.029432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b0eee-b4e6-4527-9795-5c3df1091a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:04.030189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b0eee-b4e6-4527-9795-5c3df1091a1f admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.030479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b0eee-b4e6-4527-9795-5c3df1091a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.030783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b0eee-b4e6-4527-9795-5c3df1091a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.031540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b0eee-b4e6-4527-9795-5c3df1091a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:04.033580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b0eee-b4e6-4527-9795-5c3df1091a1f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:04.137682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 925/1833] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:03 2022] POST /identity/v3/projects => generated 409 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:04.140151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.141018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.141303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.186442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:04.188408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a144e-1e77-4f20-9508-5d6e59246181 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:04.193133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:04.329215 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a912a6d-bf24-4e61-be95-721399544443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:04.332433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a912a6d-bf24-4e61-be95-721399544443 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.334951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a912a6d-bf24-4e61-be95-721399544443 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.336370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a912a6d-bf24-4e61-be95-721399544443 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.338514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a912a6d-bf24-4e61-be95-721399544443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:04.343142 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a912a6d-bf24-4e61-be95-721399544443 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:04.404838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 926/1834] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:04.407928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:04.414935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 909/1835] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 537 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:04.418725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3752b9-ec4d-40d4-9a25-a40fc1d10931 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.420865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:04.425099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3752b9-ec4d-40d4-9a25-a40fc1d10931 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.425768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3752b9-ec4d-40d4-9a25-a40fc1d10931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.453986 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee1c23-747c-4b4e-b999-e486db0cd2b3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:04.464961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-1b3752b9-ec4d-40d4-9a25-a40fc1d10931 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:04.466379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-1b3752b9-ec4d-40d4-9a25-a40fc1d10931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:04.470328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:04.471201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.471541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.471835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.472746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:04.475673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:04.502036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 910/1836] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:04.503893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3752b9-ec4d-40d4-9a25-a40fc1d10931 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:04.520362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 927/1837] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:04 2022] POST /identity/v3/users => generated 305 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:04.520362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:04.522807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3752b9-ec4d-40d4-9a25-a40fc1d10931 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:04.535111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:04.537013 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.537564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.538150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] PATH_INFO: `/v3/projects/4ca7ca5c553548c6b17e694d70f79db6/users/1522cd9a695e4975a95852fe3fce9eb7/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.539822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e3ad-eebc-48a1-a59f-d412278ea776 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:04.556256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:04.557055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.557328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.557605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.559517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:04.560093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:04.562250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:04.562975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca7ca5c553548c6b17e694d70f79db6, user_id=1522cd9a695e4975a95852fe3fce9eb7, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:04.567531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:04.589354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 928/1838] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:04.590240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 911/1839] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:04 2022] PUT /identity/v3/projects/4ca7ca5c553548c6b17e694d70f79db6/users/1522cd9a695e4975a95852fe3fce9eb7/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:04.595024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:04.599787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:04.615603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c532566-8d28-4852-918a-8046f255abcf admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:04.717394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bb2cc-f316-4dd9-b5ca-23a3b2255139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:04.718815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb2cc-f316-4dd9-b5ca-23a3b2255139 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.719206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb2cc-f316-4dd9-b5ca-23a3b2255139 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.719580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bb2cc-f316-4dd9-b5ca-23a3b2255139 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.720450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bb2cc-f316-4dd9-b5ca-23a3b2255139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:04.722344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bb2cc-f316-4dd9-b5ca-23a3b2255139 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:04.757571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 929/1840] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:04.757571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb3605-2ace-467f-9a33-d65aa6176081 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.758874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb3605-2ace-467f-9a33-d65aa6176081 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.760117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb3605-2ace-467f-9a33-d65aa6176081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.774237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0707530-582b-489e-94fa-6d84950e02be admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:04.779740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:04.780435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:04.780718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:04.781041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:04.781750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:04.784102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:04.823673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b6fb3605-2ace-467f-9a33-d65aa6176081 None None] MFA Rules not processed for user `1522cd9a695e4975a95852fe3fce9eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:04.824544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b6fb3605-2ace-467f-9a33-d65aa6176081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:04.839572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 912/1841] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:04 2022] POST /identity/v3/projects => generated 377 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:04.840954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:04.978771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 930/1842] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:04.981000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb3605-2ace-467f-9a33-d65aa6176081 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:05.005505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb3605-2ace-467f-9a33-d65aa6176081 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:05.007877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6234d50-e085-49a8-baea-3ec4d8c20d10 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:05.024484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a93d1d-f795-4479-b121-b11e37c76e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:05.029198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a93d1d-f795-4479-b121-b11e37c76e04 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.029819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a93d1d-f795-4479-b121-b11e37c76e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.030459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a93d1d-f795-4479-b121-b11e37c76e04 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.032124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a93d1d-f795-4479-b121-b11e37c76e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:05.035568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a93d1d-f795-4479-b121-b11e37c76e04 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:05.036524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:05.037999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.038498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.039060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.040546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:05.042907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:05.058425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 913/1843] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:05.060098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:05.080608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 931/1844] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:04 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:05.083073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6955e7-aab9-4071-93e7-7deaa783e126 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.083765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6955e7-aab9-4071-93e7-7deaa783e126 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.084056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6955e7-aab9-4071-93e7-7deaa783e126 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.086958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 932/1845] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:53:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:53:05.087763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5ca5f-d7fd-47e1-8b54-704fdc70f2d1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:05.089124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6955e7-aab9-4071-93e7-7deaa783e126 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:05.096126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:05.099318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.100257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.101034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.102347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:05.104479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:05.117219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6955e7-aab9-4071-93e7-7deaa783e126 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:05.135687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c337d-81a7-492c-a30f-5c10108497b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:05.137343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c337d-81a7-492c-a30f-5c10108497b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.137909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c337d-81a7-492c-a30f-5c10108497b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.138619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c337d-81a7-492c-a30f-5c10108497b3 admin admin] PATH_INFO: `/v3/projects/6a0e6946eb864024bcd6d062d96cffd8/users/9ca50812761a42339a93c179444657f9/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.174782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 914/1846] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:05 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:05.175953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c337d-81a7-492c-a30f-5c10108497b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0e6946eb864024bcd6d062d96cffd8, user_id=9ca50812761a42339a93c179444657f9, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:05.176730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:05.182284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c337d-81a7-492c-a30f-5c10108497b3 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:05.219757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 933/1847] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:05 2022] PUT /identity/v3/projects/6a0e6946eb864024bcd6d062d96cffd8/users/9ca50812761a42339a93c179444657f9/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:05.236642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed34b3b-9315-4cc2-a9be-9194a7ce3d10 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.241925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed34b3b-9315-4cc2-a9be-9194a7ce3d10 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.242407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed34b3b-9315-4cc2-a9be-9194a7ce3d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.332821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4ed34b3b-9315-4cc2-a9be-9194a7ce3d10 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:05.355274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4ed34b3b-9315-4cc2-a9be-9194a7ce3d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:05.431897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fc9de-e380-42df-967a-c933cc86459a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:05.660652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 934/1848] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:05.662037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed34b3b-9315-4cc2-a9be-9194a7ce3d10 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:05.721414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727f340-b2a4-4c5e-a0b4-7df201bbe481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:05.722035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727f340-b2a4-4c5e-a0b4-7df201bbe481 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.722317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727f340-b2a4-4c5e-a0b4-7df201bbe481 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.724973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727f340-b2a4-4c5e-a0b4-7df201bbe481 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.725687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727f340-b2a4-4c5e-a0b4-7df201bbe481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:05.732926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727f340-b2a4-4c5e-a0b4-7df201bbe481 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:05.757102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 915/1849] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 581 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:05.761910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3376c1-d593-41aa-b6e3-89174c6f19c0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.762372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3376c1-d593-41aa-b6e3-89174c6f19c0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.762942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3376c1-d593-41aa-b6e3-89174c6f19c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.833617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-bb3376c1-d593-41aa-b6e3-89174c6f19c0 None None] MFA Rules not processed for user `9ca50812761a42339a93c179444657f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:05.838851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-bb3376c1-d593-41aa-b6e3-89174c6f19c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:05.877185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed34b3b-9315-4cc2-a9be-9194a7ce3d10 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:05.897127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:05.898393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:05.898873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:05.899344 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:05.900650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:05.904254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:05.959129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 935/1850] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:05 2022] POST /identity/v3/users => generated 299 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:05.963924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:05.986977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524ca4f-4f6d-492e-b805-7c90748b22e2 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.008719 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.012728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.013734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.015029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.017247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.025096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 916/1851] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:06.025305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.025415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3376c1-d593-41aa-b6e3-89174c6f19c0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.055032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 936/1852] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:06.056357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3376c1-d593-41aa-b6e3-89174c6f19c0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.057282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.074876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.075491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.075865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.076105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.077400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.079920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.082077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e01c-2f17-44f6-9a8e-864e35291a1f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.093315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.094398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.094917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.095797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.097007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.099225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 917/1853] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:06.100685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.107304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.116831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369e908-9125-4414-9ff3-7096edd2542a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.121800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.122992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.123459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.124118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] PATH_INFO: `/v3/projects/6e1e07f7318b4fb48bafdf7b62173533/users/71a10f7b280a42ffb7672a1c629b3200/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.170001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1e07f7318b4fb48bafdf7b62173533, user_id=71a10f7b280a42ffb7672a1c629b3200, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.174686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.177947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 937/1854] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:06 2022] POST /identity/v3/projects => generated 407 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:06.178924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.209087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 918/1855] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:06 2022] PUT /identity/v3/projects/6e1e07f7318b4fb48bafdf7b62173533/users/71a10f7b280a42ffb7672a1c629b3200/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:06.210084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.356433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464330f-66ea-4060-9e1d-5f3b688da045 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.371635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.372756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.373690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.374809 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] PATH_INFO: `/v3/projects/2fea14d952e044198064f8c269b5ee8a/users/381a160cc007455bbc2c5328899dce4b/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.382798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23794b9-1f71-4791-929c-fdaffc45d3cb admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.409017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5421a-9964-4289-b058-2323114420c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.409319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5421a-9964-4289-b058-2323114420c8 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.409575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5421a-9964-4289-b058-2323114420c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.410076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5421a-9964-4289-b058-2323114420c8 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.411190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5421a-9964-4289-b058-2323114420c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.415359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5421a-9964-4289-b058-2323114420c8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.438637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fea14d952e044198064f8c269b5ee8a, user_id=381a160cc007455bbc2c5328899dce4b, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.448433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.463515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 919/1856] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:06 2022] POST /identity/v3/users => generated 310 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:06.465740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9ed75-13d1-4b56-be68-c2bb41757459 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.466043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9ed75-13d1-4b56-be68-c2bb41757459 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.466681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9ed75-13d1-4b56-be68-c2bb41757459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.495507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 938/1857] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:06 2022] PUT /identity/v3/projects/2fea14d952e044198064f8c269b5ee8a/users/381a160cc007455bbc2c5328899dce4b/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:06.495507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.504689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-faf9ed75-13d1-4b56-be68-c2bb41757459 None None] MFA Rules not processed for user `71a10f7b280a42ffb7672a1c629b3200`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:06.504689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-faf9ed75-13d1-4b56-be68-c2bb41757459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:06.628071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 920/1858] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:06.629421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9ed75-13d1-4b56-be68-c2bb41757459 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.703468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9dcd-3da3-41e6-a3c9-7801a2577852 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.712175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.713247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.713701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.714258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.715305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.719685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.741051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 939/1859] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:06.741051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.746260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:06.777566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9ed75-13d1-4b56-be68-c2bb41757459 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.804780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.805920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.806384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.806872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.807923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.810384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.858990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 921/1860] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:06.860732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.897152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b62776-eb0e-485e-95f9-fc3fef72196d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.911728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.912827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.913208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.913574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.914623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:06.915256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdda350-729b-4a5d-a5a3-7d2f5974bde5 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:06.916925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:06.963561 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:06.965221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:06.965796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:06.966432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:06.972296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 922/1861] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:06 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:06.973544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:06.996958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.002505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.020065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 940/1862] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:07.021434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.033015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15a1d0-404a-4669-8233-48394d0d523e service nova] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.091255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0255f0-bd2b-430f-b792-3adbc0e50dd6 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.105548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14ae59-46ae-4e95-aea2-dc0d4a0485f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.108912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ae59-46ae-4e95-aea2-dc0d4a0485f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.110698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ae59-46ae-4e95-aea2-dc0d4a0485f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.110984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14ae59-46ae-4e95-aea2-dc0d4a0485f3 admin admin] PATH_INFO: `/v3/projects/e73c8c35aec742b4b312082160b29290/users/6ce74fb513c142a09f66546dbdb30e3e/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.130249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.131395 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.131754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.132079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] PATH_INFO: `/v3/projects/2fea14d952e044198064f8c269b5ee8a/users/381a160cc007455bbc2c5328899dce4b/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.134600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fea14d952e044198064f8c269b5ee8a, user_id=381a160cc007455bbc2c5328899dce4b, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.137825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.144299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14ae59-46ae-4e95-aea2-dc0d4a0485f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73c8c35aec742b4b312082160b29290, user_id=6ce74fb513c142a09f66546dbdb30e3e, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.148130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14ae59-46ae-4e95-aea2-dc0d4a0485f3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.164855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 941/1863] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:07 2022] PUT /identity/v3/projects/2fea14d952e044198064f8c269b5ee8a/users/381a160cc007455bbc2c5328899dce4b/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:07.169021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.178114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 923/1864] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:06 2022] PUT /identity/v3/projects/e73c8c35aec742b4b312082160b29290/users/6ce74fb513c142a09f66546dbdb30e3e/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:07.180055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdabcd-0dc8-477b-a5ba-2e4443fbd057 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.180469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdabcd-0dc8-477b-a5ba-2e4443fbd057 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.181043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdabcd-0dc8-477b-a5ba-2e4443fbd057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.220195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-9ccdabcd-0dc8-477b-a5ba-2e4443fbd057 None None] MFA Rules not processed for user `381a160cc007455bbc2c5328899dce4b`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:07.221736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9ccdabcd-0dc8-477b-a5ba-2e4443fbd057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:07.266715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acbd85-7577-4491-8d60-46cde6ac9fbf admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.313000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 924/1865] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:07.314264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdabcd-0dc8-477b-a5ba-2e4443fbd057 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.419629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.421844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.423153 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.423647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.425156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.428956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.470389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 942/1866] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:07 2022] POST /identity/v3/users => generated 295 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:07.472088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.475496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:07.488145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdabcd-0dc8-477b-a5ba-2e4443fbd057 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.495706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.496569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.496901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.497753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.498660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.500347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.514852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 925/1867] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:07.515857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.528525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ae3f9-e17a-45d6-852c-8c7dd96a4f6f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.535605 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.536335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.536598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.536931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.537674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.539452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.553550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 926/1868] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:07.555665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.570679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e292ae-0311-4b08-aa5f-51a41e6fcb4e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.578489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.580360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.581692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563aee3-95f1-4f17-9197-81f6ab72cd6d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.582059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.582532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] PATH_INFO: `/v3/projects/e73c8c35aec742b4b312082160b29290/users/6ce74fb513c142a09f66546dbdb30e3e/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.585893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73c8c35aec742b4b312082160b29290, user_id=6ce74fb513c142a09f66546dbdb30e3e, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.589253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.592559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ef400-4be3-4a46-a8b4-872750e38895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.593334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ef400-4be3-4a46-a8b4-872750e38895 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.593676 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ef400-4be3-4a46-a8b4-872750e38895 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.594187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ef400-4be3-4a46-a8b4-872750e38895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.605838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 927/1869] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:07 2022] PUT /identity/v3/projects/e73c8c35aec742b4b312082160b29290/users/6ce74fb513c142a09f66546dbdb30e3e/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:07.606904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.611250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ef400-4be3-4a46-a8b4-872750e38895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.613305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ef400-4be3-4a46-a8b4-872750e38895 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.648567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 943/1870] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:07.651842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.652402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.652972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.702685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbfb41-8acd-4ae9-b9a3-21f7746cf018 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.717848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] MFA Rules not processed for user `6ce74fb513c142a09f66546dbdb30e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:07.720440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:07.747647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.754569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.757312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.758633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] PATH_INFO: `/v3/projects/d4095820724c4470a4a5d655e192b502/users/d6834f8521044600a955554d1c2d6f33/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.794403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4095820724c4470a4a5d655e192b502, user_id=d6834f8521044600a955554d1c2d6f33, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.797701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.827541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 928/1871] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:07 2022] PUT /identity/v3/projects/d4095820724c4470a4a5d655e192b502/users/d6834f8521044600a955554d1c2d6f33/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:07.828878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.924358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cb20b-8819-4861-97d6-c27953450491 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:07.931346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 944/1872] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:07.939264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:07.940089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:07.940500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:07.941090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:07.942866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:07.946837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:07.947731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.952259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:07.965413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 929/1873] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:53:07.967414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:07.990289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e5202-5c38-42e7-a2ee-8c33f47f6f07 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:08.002921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb42901-acee-4af4-bf08-006ebc31764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:08.005337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42901-acee-4af4-bf08-006ebc31764a admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:08.006274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42901-acee-4af4-bf08-006ebc31764a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:08.006877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb42901-acee-4af4-bf08-006ebc31764a admin admin] PATH_INFO: `/v3/projects/d4095820724c4470a4a5d655e192b502/users/d6834f8521044600a955554d1c2d6f33/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:08.027458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb42901-acee-4af4-bf08-006ebc31764a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4095820724c4470a4a5d655e192b502, user_id=d6834f8521044600a955554d1c2d6f33, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:08.043036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb42901-acee-4af4-bf08-006ebc31764a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:08.071068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ce59b-6648-41ef-ab38-d4b168ca9b38 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:08.084919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 930/1874] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:07 2022] PUT /identity/v3/projects/d4095820724c4470a4a5d655e192b502/users/d6834f8521044600a955554d1c2d6f33/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:08.093009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597df7a-9e4e-4d4b-b522-68049044cf0b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:08.093566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597df7a-9e4e-4d4b-b522-68049044cf0b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:08.093945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597df7a-9e4e-4d4b-b522-68049044cf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:08.140380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-9597df7a-9e4e-4d4b-b522-68049044cf0b None None] MFA Rules not processed for user `d6834f8521044600a955554d1c2d6f33`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:08.143393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9597df7a-9e4e-4d4b-b522-68049044cf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:08.244946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:08.245666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:08.245949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:08.246225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:08.296180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 931/1875] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:08.327532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597df7a-9e4e-4d4b-b522-68049044cf0b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:08.352354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597df7a-9e4e-4d4b-b522-68049044cf0b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:08.362828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:08.363901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:08.364902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:08.381014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:08.382577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:08.385114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:08.404106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 945/1876] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:08.486671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:08.488208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:08.505239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 932/1877] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:08.916840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:08.992203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5f15f-d59f-4e18-bc14-f56a5f391d22 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:09.009331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:09.016133 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:09.016612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:09.017027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:09.260403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:09.263698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:09.283552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 946/1878] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:09.374086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:09.383559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fdcd9-50c5-4c6e-9934-9068445e0815 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:09.387953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:09.388735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:09.389448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:09.389748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:09.394909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:09.472245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:09.475288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:09.487111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 933/1879] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:13.689378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:13.699430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0451b-55d3-4e45-91e6-f6e23abaf7ce service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:13.703789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8bb14-c559-475b-94ec-214a74b0cb2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:13.704849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8bb14-c559-475b-94ec-214a74b0cb2e service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:13.705345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8bb14-c559-475b-94ec-214a74b0cb2e service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:13.705828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8bb14-c559-475b-94ec-214a74b0cb2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:13.717410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8bb14-c559-475b-94ec-214a74b0cb2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:13.719319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec8bb14-c559-475b-94ec-214a74b0cb2e service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:13.743553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 947/1880] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:21.319447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:21.436042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0991f8-4d4b-4025-9cb9-4b73c58e5a86 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:21.440671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:21.441674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:21.441963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:21.442262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] PATH_INFO: `/v3/users/9ca50812761a42339a93c179444657f9` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:21.443503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca50812761a42339a93c179444657f9)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:21.445512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:21.502107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca50812761a42339a93c179444657f9'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:21.502740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca50812761a42339a93c179444657f9'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:21.503458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca50812761a42339a93c179444657f9'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:21.516162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca50812761a42339a93c179444657f9'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:21.532818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca50812761a42339a93c179444657f9'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:21.549460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 934/1881] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:21 2022] DELETE /identity/v3/users/9ca50812761a42339a93c179444657f9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:21.555161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:21.555915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:21.555915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:21.575850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:21.577615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:21.651754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 948/1882] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:21.659114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:21.661045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:21.752479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5a537-9593-4fd4-af51-4071d8ece584 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:21.768378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:21.769133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:21.770212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:21.770503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:21.782610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:21.784060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:21.797578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 935/1883] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:22.109534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:22.111013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:22.123688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788287e7-006a-4d90-ad7d-ef5f328229ff None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:22.131053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:22.132178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:22.132512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:22.133014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] PATH_INFO: `/v3/projects/6a0e6946eb864024bcd6d062d96cffd8` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:22.134539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a0e6946eb864024bcd6d062d96cffd8)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:22.136511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:22.206020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a0e6946eb864024bcd6d062d96cffd8', 'request_id': 'req-b5faa91d-ce47-4d85-95db-f6839722f9c9'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.218022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a0e6946eb864024bcd6d062d96cffd8', 'request_id': 'req-b5faa91d-ce47-4d85-95db-f6839722f9c9'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.228736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a0e6946eb864024bcd6d062d96cffd8', 'request_id': 'req-b5faa91d-ce47-4d85-95db-f6839722f9c9'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.243237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] The token cache is being invalidate because project 6a0e6946eb864024bcd6d062d96cffd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:22.243545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.244317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.248913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 949/1884] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:22 2022] DELETE /identity/v3/projects/6a0e6946eb864024bcd6d062d96cffd8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:53:22.251250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:22.268972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf17e7-c1c5-47d2-9733-eef05b0ea4ef service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:22.347626 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:22.349411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:22.349944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:22.350441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] PATH_INFO: `/v3/users/6ce74fb513c142a09f66546dbdb30e3e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:22.352737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ce74fb513c142a09f66546dbdb30e3e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:22.356529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:22.422226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ce74fb513c142a09f66546dbdb30e3e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.423019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ce74fb513c142a09f66546dbdb30e3e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.424428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ce74fb513c142a09f66546dbdb30e3e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.437813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ce74fb513c142a09f66546dbdb30e3e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.470509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ce74fb513c142a09f66546dbdb30e3e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:22.491722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 936/1885] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:22 2022] DELETE /identity/v3/users/6ce74fb513c142a09f66546dbdb30e3e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:22.561854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:22.563957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:22.658825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faa91d-ce47-4d85-95db-f6839722f9c9 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:22.737314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac7b30-7b85-49d4-a70c-a55b234c43d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:22.738120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac7b30-7b85-49d4-a70c-a55b234c43d5 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:22.738499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac7b30-7b85-49d4-a70c-a55b234c43d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:22.739048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac7b30-7b85-49d4-a70c-a55b234c43d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:22.744424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-41ac7b30-7b85-49d4-a70c-a55b234c43d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:22.831020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac7b30-7b85-49d4-a70c-a55b234c43d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:22.832669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac7b30-7b85-49d4-a70c-a55b234c43d5 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:22.844187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 950/1886] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:23.016843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.018417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:23.108433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f4ffd-5499-4674-9d52-6137f29e5974 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.113007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.113873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.114162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.114437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] PATH_INFO: `/v3/projects/e73c8c35aec742b4b312082160b29290` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.115790 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73c8c35aec742b4b312082160b29290)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.117508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.168882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73c8c35aec742b4b312082160b29290', 'request_id': 'req-e5984951-77a5-4b27-abd8-6579505c55e8'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:23.175280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73c8c35aec742b4b312082160b29290', 'request_id': 'req-e5984951-77a5-4b27-abd8-6579505c55e8'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:23.180964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73c8c35aec742b4b312082160b29290', 'request_id': 'req-e5984951-77a5-4b27-abd8-6579505c55e8'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:23.193920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] The token cache is being invalidate because project e73c8c35aec742b4b312082160b29290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:23.194107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:23.194725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5984951-77a5-4b27-abd8-6579505c55e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:23.196306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 937/1887] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:23 2022] DELETE /identity/v3/projects/e73c8c35aec742b4b312082160b29290 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:23.202268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65fc92-4128-4d86-bb75-5675e6676b32 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.202555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65fc92-4128-4d86-bb75-5675e6676b32 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.202866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65fc92-4128-4d86-bb75-5675e6676b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.221832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ff65fc92-4128-4d86-bb75-5675e6676b32 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:23.223229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ff65fc92-4128-4d86-bb75-5675e6676b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:23.291949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 951/1888] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:23.305773 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb211e5c-dd21-4f70-88fa-b9e886edaf8b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.306703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb211e5c-dd21-4f70-88fa-b9e886edaf8b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.306703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb211e5c-dd21-4f70-88fa-b9e886edaf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.336292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-fb211e5c-dd21-4f70-88fa-b9e886edaf8b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:23.337753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fb211e5c-dd21-4f70-88fa-b9e886edaf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:23.351468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 938/1889] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:23.357065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65fc92-4128-4d86-bb75-5675e6676b32 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.372390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65fc92-4128-4d86-bb75-5675e6676b32 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.377443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.378324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.378833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.379193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.379876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:23.380367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.382062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.396848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 952/1890] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:23.400852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb211e5c-dd21-4f70-88fa-b9e886edaf8b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.410339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb211e5c-dd21-4f70-88fa-b9e886edaf8b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.415135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.415811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.416147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.416853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.417196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.419223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.464696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 939/1891] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:23 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:23.468137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.480310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757238f3-ee32-4669-9c31-7aa29f83ca48 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.556027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.556566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.556876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.557144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.557864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.560226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.585261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 953/1892] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:23 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:23.589432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.602089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d017d1-bc36-452e-b24b-b1a2c4507bc2 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.607345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.608057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.608342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.608621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.609406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.611016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.622813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 940/1893] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:23.635397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.652786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f564b6-fd4b-42bb-b99e-a0fb57eb68c9 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.665104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7c6bc-7a4d-4fbe-8f06-f8e5a3cfc2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.666157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7c6bc-7a4d-4fbe-8f06-f8e5a3cfc2aa admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.666584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7c6bc-7a4d-4fbe-8f06-f8e5a3cfc2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.667215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7c6bc-7a4d-4fbe-8f06-f8e5a3cfc2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.668289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7c6bc-7a4d-4fbe-8f06-f8e5a3cfc2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.670588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7c6bc-7a4d-4fbe-8f06-f8e5a3cfc2aa admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.695802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 954/1894] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:23.697945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.709027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5463d00-2b6a-425c-a354-38fe3d424872 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.714003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.715126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.715544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.715977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] PATH_INFO: `/v3/projects/b0738a12e3794e2b9a45beb56d6eeb09/users/ddff5d5e6a5643529cae5f6cb075bd9e/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.740264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0738a12e3794e2b9a45beb56d6eeb09, user_id=ddff5d5e6a5643529cae5f6cb075bd9e, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.743231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.777448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 941/1895] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:23 2022] PUT /identity/v3/projects/b0738a12e3794e2b9a45beb56d6eeb09/users/ddff5d5e6a5643529cae5f6cb075bd9e/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:23.781433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.781748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.782026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.819510 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] MFA Rules not processed for user `ddff5d5e6a5643529cae5f6cb075bd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:23.820829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:23.888635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 955/1896] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:23.900472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:23.902299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:23.964710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3566-f946-4bef-93bc-46657b11b818 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:23.969495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15099471-e231-4c0b-94c5-f5c530cdd8a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:23.970313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15099471-e231-4c0b-94c5-f5c530cdd8a1 service glance] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:23.970709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15099471-e231-4c0b-94c5-f5c530cdd8a1 service glance] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:23.971060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15099471-e231-4c0b-94c5-f5c530cdd8a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:23.980871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15099471-e231-4c0b-94c5-f5c530cdd8a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:23.982755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15099471-e231-4c0b-94c5-f5c530cdd8a1 service glance] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:23.993733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 942/1897] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:26.481584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:26.493647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:26.571684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e18918-aac4-417b-bc4a-975b986a253f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:26.577836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:26.578900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:26.580725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:26.581318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] PATH_INFO: `/v3/users/a4128309c66e4147a922956161ccfc36` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:26.582391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4128309c66e4147a922956161ccfc36)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:26.584465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:26.645365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4128309c66e4147a922956161ccfc36'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:26.658587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4128309c66e4147a922956161ccfc36'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:26.659280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4128309c66e4147a922956161ccfc36'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:26.659870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4128309c66e4147a922956161ccfc36'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:26.672735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4128309c66e4147a922956161ccfc36'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:26.686249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 956/1898] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:26 2022] DELETE /identity/v3/users/a4128309c66e4147a922956161ccfc36 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:26.690966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:26.691454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:26.691543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:26.709886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:26.711105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:26.775975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 943/1899] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:26.786054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:26.787751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:26.860426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1182e0-0629-422b-be46-8551399dc40f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:26.864264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:26.865144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:26.865487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:26.865825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:26.876091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:26.877593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:26.891888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 957/1900] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:27.201833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:27.204263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:27.222998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb10c6-b4a3-4112-a035-637288c904d6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:27.228833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:27.229934 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:27.230414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:27.230678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] PATH_INFO: `/v3/projects/0ae81a31e30e4e338973f430b1cf2c3c` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:27.232119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae81a31e30e4e338973f430b1cf2c3c)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:27.234182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:27.298997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae81a31e30e4e338973f430b1cf2c3c', 'request_id': 'req-e5659288-d951-4ff7-a5de-682c40f9724c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.306877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae81a31e30e4e338973f430b1cf2c3c', 'request_id': 'req-e5659288-d951-4ff7-a5de-682c40f9724c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.325593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae81a31e30e4e338973f430b1cf2c3c', 'request_id': 'req-e5659288-d951-4ff7-a5de-682c40f9724c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.343748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] The token cache is being invalidate because project 0ae81a31e30e4e338973f430b1cf2c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:27.344041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.344668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.346359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 944/1901] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:27 2022] DELETE /identity/v3/projects/0ae81a31e30e4e338973f430b1cf2c3c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:27.351241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:27.362940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f42e7-ddcf-490f-9f18-5503d68e7e19 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:27.450219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:27.451302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:27.451761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:27.452330 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] PATH_INFO: `/v3/users/1522cd9a695e4975a95852fe3fce9eb7` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:27.454387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1522cd9a695e4975a95852fe3fce9eb7)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:27.458007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:27.528756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1522cd9a695e4975a95852fe3fce9eb7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.549232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1522cd9a695e4975a95852fe3fce9eb7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.550303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1522cd9a695e4975a95852fe3fce9eb7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.551093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1522cd9a695e4975a95852fe3fce9eb7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.575627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1522cd9a695e4975a95852fe3fce9eb7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:27.608707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 958/1902] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:27 2022] DELETE /identity/v3/users/1522cd9a695e4975a95852fe3fce9eb7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:28.442106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:28.443678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:28.485496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5659288-d951-4ff7-a5de-682c40f9724c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:28.581366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:28.582927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:28.583463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:28.583912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] PATH_INFO: `/v3/projects/4ca7ca5c553548c6b17e694d70f79db6` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:28.586282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca7ca5c553548c6b17e694d70f79db6)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:28.588974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:28.649839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca7ca5c553548c6b17e694d70f79db6', 'request_id': 'req-a3cbd1e2-35b8-42c9-8819-5043844e9cad'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.657385 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca7ca5c553548c6b17e694d70f79db6', 'request_id': 'req-a3cbd1e2-35b8-42c9-8819-5043844e9cad'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.664064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca7ca5c553548c6b17e694d70f79db6', 'request_id': 'req-a3cbd1e2-35b8-42c9-8819-5043844e9cad'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.678455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] The token cache is being invalidate because project 4ca7ca5c553548c6b17e694d70f79db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:28.678731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.679386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.682043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 945/1903] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:28 2022] DELETE /identity/v3/projects/4ca7ca5c553548c6b17e694d70f79db6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:28.686625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:28.688197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:28.702165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b49912-8244-4682-a864-553fd72b671e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:28.775557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:28.776993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:28.777501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:28.778053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] PATH_INFO: `/v3/users/381a160cc007455bbc2c5328899dce4b` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:28.779820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=381a160cc007455bbc2c5328899dce4b)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:28.782678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:28.867211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381a160cc007455bbc2c5328899dce4b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.889361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381a160cc007455bbc2c5328899dce4b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.890342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381a160cc007455bbc2c5328899dce4b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.891296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381a160cc007455bbc2c5328899dce4b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.904183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d959790c-4463-4280-ab58-4fc381d28fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381a160cc007455bbc2c5328899dce4b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:28.923413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 959/1904] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:28 2022] DELETE /identity/v3/users/381a160cc007455bbc2c5328899dce4b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:29.353090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:29.354762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:29.367896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbd1e2-35b8-42c9-8819-5043844e9cad admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:29.455270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:29.456573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:29.456900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:29.457184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] PATH_INFO: `/v3/projects/2fea14d952e044198064f8c269b5ee8a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:29.458777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fea14d952e044198064f8c269b5ee8a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:29.460738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:29.530325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fea14d952e044198064f8c269b5ee8a', 'request_id': 'req-dbc5458c-3300-4d3c-a9f7-61528cea7293'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:29.538999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fea14d952e044198064f8c269b5ee8a', 'request_id': 'req-dbc5458c-3300-4d3c-a9f7-61528cea7293'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:29.550549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fea14d952e044198064f8c269b5ee8a', 'request_id': 'req-dbc5458c-3300-4d3c-a9f7-61528cea7293'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:29.571131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] The token cache is being invalidate because project 2fea14d952e044198064f8c269b5ee8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:29.572015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:29.573386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-dbc5458c-3300-4d3c-a9f7-61528cea7293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:29.582441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 946/1905] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:29 2022] DELETE /identity/v3/projects/2fea14d952e044198064f8c269b5ee8a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:29.587700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d6273-aaf9-498a-8e86-193a8d772c76 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:29.587700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d6273-aaf9-498a-8e86-193a8d772c76 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:29.588012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d6273-aaf9-498a-8e86-193a8d772c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:29.615946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c19d6273-aaf9-498a-8e86-193a8d772c76 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:29.617301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c19d6273-aaf9-498a-8e86-193a8d772c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:29.697632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 960/1906] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:29.708114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238fe0-5495-4d4d-a5ff-614527cc188f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:29.708708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238fe0-5495-4d4d-a5ff-614527cc188f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:29.709327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee238fe0-5495-4d4d-a5ff-614527cc188f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:29.739620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ee238fe0-5495-4d4d-a5ff-614527cc188f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:29.742067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ee238fe0-5495-4d4d-a5ff-614527cc188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:29.756269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 947/1907] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:29.762448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d6273-aaf9-498a-8e86-193a8d772c76 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:29.773804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d6273-aaf9-498a-8e86-193a8d772c76 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:29.780610 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:29.782596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:29.782596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:29.782596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:29.783589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:29.783589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:29.785858 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:29.800650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 961/1908] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:29.805331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238fe0-5495-4d4d-a5ff-614527cc188f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:29.816459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee238fe0-5495-4d4d-a5ff-614527cc188f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:29.821732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:29.822668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:29.822983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:29.823325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:29.824116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:29.826209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:29.857021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 948/1909] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:29 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:29.864002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:29.874632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ff68-598c-419a-96f2-d7fb14a79124 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:29.941431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:29.942279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:29.942559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:29.942869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:29.943608 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:29.945594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:29.972435 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 962/1910] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:29 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:29.973945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:29.993545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f59c-a3b4-44aa-8d6f-5b572e79d014 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.004917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.005638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.005969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.006194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.006966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.009338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.024861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 949/1911] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:30.027798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.044611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851390-d142-43ea-90b8-2b5a2bc54270 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.051089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1b116-ff4b-4c5b-ae8e-073bd1a88c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.051810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1b116-ff4b-4c5b-ae8e-073bd1a88c7d admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.052257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1b116-ff4b-4c5b-ae8e-073bd1a88c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.052419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1b116-ff4b-4c5b-ae8e-073bd1a88c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.053217 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1b116-ff4b-4c5b-ae8e-073bd1a88c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.055131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1b116-ff4b-4c5b-ae8e-073bd1a88c7d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.071294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 963/1912] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:30.076467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.087888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873ef62-f760-48da-b0e5-f919df47ceea admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.094004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.094938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.095217 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.095496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] PATH_INFO: `/v3/projects/8b5424c0e34546a9b25c77a1da572539/users/69ce0f189056479a8e29a87403bf4e56/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.112331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5424c0e34546a9b25c77a1da572539, user_id=69ce0f189056479a8e29a87403bf4e56, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.115114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.132274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 950/1913] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:30 2022] PUT /identity/v3/projects/8b5424c0e34546a9b25c77a1da572539/users/69ce0f189056479a8e29a87403bf4e56/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:30.145122 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4b0c7-3b70-495a-98f8-ec987bbe382a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.145421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4b0c7-3b70-495a-98f8-ec987bbe382a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.145688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4b0c7-3b70-495a-98f8-ec987bbe382a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.175876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-65e4b0c7-3b70-495a-98f8-ec987bbe382a None None] MFA Rules not processed for user `69ce0f189056479a8e29a87403bf4e56`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:30.177205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-65e4b0c7-3b70-495a-98f8-ec987bbe382a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:30.251908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 964/1914] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:30.258064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.334010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000f4a7-aba0-43fa-bf7c-35af02f01c03 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.340316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.341628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.341921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.342308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.343065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.345049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.375129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 951/1915] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:30 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:30.381235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4b0c7-3b70-495a-98f8-ec987bbe382a None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.396153 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4b0c7-3b70-495a-98f8-ec987bbe382a None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.480276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.481259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.481526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.481856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.483262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.484900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.516649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 965/1916] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:30 2022] POST /identity/v3/users => generated 302 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:30.517945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.529951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef33ba7-dbbe-48af-bb95-a8299ff5a746 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.536118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.536875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.537164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.537439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.538237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.540019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.551804 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 952/1917] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:30.556407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.568515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc008fe-b4d1-446b-808e-1735ff83a838 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.574283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.575364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.575644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.575931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] PATH_INFO: `/v3/projects/5b1610e53fc0488d8832bc5beca96248/users/458121addf0f4da6b723eb0636034029/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.592580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1610e53fc0488d8832bc5beca96248, user_id=458121addf0f4da6b723eb0636034029, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.595429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.612264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 966/1918] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:30 2022] PUT /identity/v3/projects/5b1610e53fc0488d8832bc5beca96248/users/458121addf0f4da6b723eb0636034029/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:30.619014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.636401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77801aa-bb08-432b-b13c-481c955a2ed2 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.709936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f0451-c07e-4aa2-9e05-c83df0fb0703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.710614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f0451-c07e-4aa2-9e05-c83df0fb0703 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.710893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f0451-c07e-4aa2-9e05-c83df0fb0703 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.711164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f0451-c07e-4aa2-9e05-c83df0fb0703 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.711939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f0451-c07e-4aa2-9e05-c83df0fb0703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.713647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f0451-c07e-4aa2-9e05-c83df0fb0703 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.724439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 953/1919] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:30.729435 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.741756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f2a80-7e34-4032-9471-54caafa7e2b0 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.746494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.747399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.747673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.748093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] PATH_INFO: `/v3/projects/5b1610e53fc0488d8832bc5beca96248/users/458121addf0f4da6b723eb0636034029/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.750089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1610e53fc0488d8832bc5beca96248, user_id=458121addf0f4da6b723eb0636034029, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:30.752735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:30.769458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 967/1920] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:30 2022] PUT /identity/v3/projects/5b1610e53fc0488d8832bc5beca96248/users/458121addf0f4da6b723eb0636034029/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:30.783614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.784022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.784518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:30.818958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] MFA Rules not processed for user `458121addf0f4da6b723eb0636034029`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:30.820209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:30.908850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 954/1921] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:30.924253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:30.926410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:30.989375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbe71f-be4c-4227-9528-c91739d88f06 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:30.993391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:30.993976 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:30.994268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:30.994538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:31.005124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:31.006618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:31.018457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 968/1922] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:31.070102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:31.071633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:31.084632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68540bb-74d9-42db-99e2-2c64aae74a29 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:31.089229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da19ae-a424-4b49-bbf8-e12a9ad0d602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:31.090723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da19ae-a424-4b49-bbf8-e12a9ad0d602 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:31.091091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da19ae-a424-4b49-bbf8-e12a9ad0d602 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:31.091434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da19ae-a424-4b49-bbf8-e12a9ad0d602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:31.166059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da19ae-a424-4b49-bbf8-e12a9ad0d602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:31.168596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46da19ae-a424-4b49-bbf8-e12a9ad0d602 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:31.183134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 955/1923] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:38.270216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:38.417464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb6a70-d92c-46b3-b350-04e7e32c9b88 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:38.424014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:38.424980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:38.425144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:38.425551 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] PATH_INFO: `/v3/users/69ce0f189056479a8e29a87403bf4e56` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:38.426900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ce0f189056479a8e29a87403bf4e56)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:38.441128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:38.494541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ce0f189056479a8e29a87403bf4e56'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:38.511568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ce0f189056479a8e29a87403bf4e56'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:38.512681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ce0f189056479a8e29a87403bf4e56'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:38.513874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ce0f189056479a8e29a87403bf4e56'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:38.525243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ce0f189056479a8e29a87403bf4e56'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:38.553084 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 969/1924] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:38 2022] DELETE /identity/v3/users/69ce0f189056479a8e29a87403bf4e56 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:38.556218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:38.557786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:38.558500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:38.590407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:38.592885 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:38.742312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 956/1925] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:38.759541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:38.761462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:38.880570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704fe7a-bc63-46ae-b678-6cd6b06a4c18 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:38.886572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:38.887470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:38.887856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:38.888117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:38.901106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:38.906325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:38.920925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 970/1926] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:39.262720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:39.265840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:39.294282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f4399-f969-45c3-937e-018413329e8e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:39.302076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:39.303143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:39.303523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:39.303881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] PATH_INFO: `/v3/projects/8b5424c0e34546a9b25c77a1da572539` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:39.305398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5424c0e34546a9b25c77a1da572539)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:39.307545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:39.379945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5424c0e34546a9b25c77a1da572539', 'request_id': 'req-68a11230-5f30-4566-9b3b-7f58a624256b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.393476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5424c0e34546a9b25c77a1da572539', 'request_id': 'req-68a11230-5f30-4566-9b3b-7f58a624256b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.403474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5424c0e34546a9b25c77a1da572539', 'request_id': 'req-68a11230-5f30-4566-9b3b-7f58a624256b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.421010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] The token cache is being invalidate because project 8b5424c0e34546a9b25c77a1da572539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:39.421363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.422549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.424025 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 957/1927] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:39 2022] DELETE /identity/v3/projects/8b5424c0e34546a9b25c77a1da572539 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:39.427914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:39.440967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7faa4-b013-4934-b152-16ac811579bb service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:39.528388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:39.529543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:39.529891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:39.530240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] PATH_INFO: `/v3/users/458121addf0f4da6b723eb0636034029` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:39.531918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=458121addf0f4da6b723eb0636034029)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:39.534258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:39.595262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458121addf0f4da6b723eb0636034029'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.610240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458121addf0f4da6b723eb0636034029'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.611079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458121addf0f4da6b723eb0636034029'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.611745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458121addf0f4da6b723eb0636034029'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.624385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3132eb95-c9ac-451f-9f1b-4f11684169f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458121addf0f4da6b723eb0636034029'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:39.664870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 971/1928] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:39 2022] DELETE /identity/v3/users/458121addf0f4da6b723eb0636034029 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:40.619678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:40.622134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:40.644871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a11230-5f30-4566-9b3b-7f58a624256b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:40.746471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:40.748769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:40.749304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:40.749797 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] PATH_INFO: `/v3/projects/5b1610e53fc0488d8832bc5beca96248` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:40.763021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1610e53fc0488d8832bc5beca96248)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:40.773482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:40.940752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1610e53fc0488d8832bc5beca96248', 'request_id': 'req-a906df4b-acf7-4a27-a189-a1ee3874f195'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:40.949956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1610e53fc0488d8832bc5beca96248', 'request_id': 'req-a906df4b-acf7-4a27-a189-a1ee3874f195'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:40.957541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1610e53fc0488d8832bc5beca96248', 'request_id': 'req-a906df4b-acf7-4a27-a189-a1ee3874f195'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:40.979217 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] The token cache is being invalidate because project 5b1610e53fc0488d8832bc5beca96248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:40.979740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:40.980811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a906df4b-acf7-4a27-a189-a1ee3874f195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:40.983912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 958/1929] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:40 2022] DELETE /identity/v3/projects/5b1610e53fc0488d8832bc5beca96248 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:40.989630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a29695-c6fb-43f9-8534-b216700601cf None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:40.990383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a29695-c6fb-43f9-8534-b216700601cf None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:40.990862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a29695-c6fb-43f9-8534-b216700601cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.022708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-56a29695-c6fb-43f9-8534-b216700601cf None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:41.024170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-56a29695-c6fb-43f9-8534-b216700601cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:41.126057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 972/1930] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:41.134448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3632-f9a1-47a9-81ea-6a084960cf93 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.134837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3632-f9a1-47a9-81ea-6a084960cf93 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.135430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b3632-f9a1-47a9-81ea-6a084960cf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.159456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ce4b3632-f9a1-47a9-81ea-6a084960cf93 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:41.168880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ce4b3632-f9a1-47a9-81ea-6a084960cf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:41.181824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 959/1931] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:41.190504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a29695-c6fb-43f9-8534-b216700601cf None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.202528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a29695-c6fb-43f9-8534-b216700601cf None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:41.209552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:41.210345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.210625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.210995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.211744 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:41.212062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:41.214030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:41.236727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 973/1932] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:41.241369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3632-f9a1-47a9-81ea-6a084960cf93 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.253957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b3632-f9a1-47a9-81ea-6a084960cf93 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:41.262728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:41.263835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.264165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.264437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.265192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:41.267302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:41.305401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 960/1933] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:41 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:41.310090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.322044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452f0a3-7929-483c-b239-3536d2ef612f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:41.422678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:41.423368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.423722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.424265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.425140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:41.427580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:41.461383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 974/1934] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:41 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:53:41.467235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.478794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397be7e1-84eb-4f66-8060-89226ffa1901 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:41.483983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:41.484719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.485039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.485318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.486106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:41.488678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:41.505032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 961/1935] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:53:41.508877 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.521318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268744d-88b5-4dac-9c1b-bc9535c46282 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:41.527430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86555d0-2229-4464-989c-48d6936a5444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:41.528649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86555d0-2229-4464-989c-48d6936a5444 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.528980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86555d0-2229-4464-989c-48d6936a5444 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.529340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86555d0-2229-4464-989c-48d6936a5444 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.530217 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86555d0-2229-4464-989c-48d6936a5444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:41.532432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86555d0-2229-4464-989c-48d6936a5444 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:41.544385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 975/1936] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:41.548827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.561136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445eda8-c719-4056-adc1-b83d55fcd7bc admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:41.566716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:41.567706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.568055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.568392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] PATH_INFO: `/v3/projects/08bf5349308b419d80b8d0076a7d467f/users/9dbc2b4ecfab411fafceb1fee0a6355d/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.585467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bf5349308b419d80b8d0076a7d467f, user_id=9dbc2b4ecfab411fafceb1fee0a6355d, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:41.588130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:41.603595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 962/1937] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:41 2022] PUT /identity/v3/projects/08bf5349308b419d80b8d0076a7d467f/users/9dbc2b4ecfab411fafceb1fee0a6355d/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:41.616093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bf514-024f-4825-8cb2-76f2e1bb2663 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.616462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bf514-024f-4825-8cb2-76f2e1bb2663 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.616911 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bf514-024f-4825-8cb2-76f2e1bb2663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.653581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-875bf514-024f-4825-8cb2-76f2e1bb2663 None None] MFA Rules not processed for user `9dbc2b4ecfab411fafceb1fee0a6355d`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:41.655284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-875bf514-024f-4825-8cb2-76f2e1bb2663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:41.799764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 976/1938] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:41.801724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.882746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd04d3-19b3-42d2-87e9-2b25e2f72d49 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:41.887820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:41.888986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:41.889270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:41.889567 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:41.890284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:41.892237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:41.920951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 963/1939] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:41 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:41.924939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bf514-024f-4825-8cb2-76f2e1bb2663 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:41.936582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bf514-024f-4825-8cb2-76f2e1bb2663 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:42.006730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:42.006730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:42.006730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:42.006730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:42.007996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:42.007996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:42.030441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 977/1940] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:41 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:53:42.034393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:42.045933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d7b01-76d5-4839-babd-99257227a776 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:42.051789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:42.052928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:42.052928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:42.053388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:42.053769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:42.055449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:42.072338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 964/1941] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:42.076616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:42.092293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090fef9-e8b7-4374-8bdd-7356d6e527b5 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:42.097476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:42.098560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:42.098841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:42.099115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] PATH_INFO: `/v3/projects/8b116b8e4fca4ad794574e0e8f8eafb5/users/aadee117c3c04af39d031b35d96cf656/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:42.125994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b116b8e4fca4ad794574e0e8f8eafb5, user_id=aadee117c3c04af39d031b35d96cf656, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:42.134130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:42.162935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 978/1942] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:42 2022] PUT /identity/v3/projects/8b116b8e4fca4ad794574e0e8f8eafb5/users/aadee117c3c04af39d031b35d96cf656/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:42.166136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:42.189228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ff942-99ce-40cd-a339-de1747fd302d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:42.259308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5555015-b92f-498a-b236-f03e2b40fb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:42.259998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5555015-b92f-498a-b236-f03e2b40fb43 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:42.260287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5555015-b92f-498a-b236-f03e2b40fb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:42.260596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5555015-b92f-498a-b236-f03e2b40fb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:42.261324 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5555015-b92f-498a-b236-f03e2b40fb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:42.262904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5555015-b92f-498a-b236-f03e2b40fb43 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:42.273838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 965/1943] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:42.278167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:42.290113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd21227-2e57-4a0b-8caf-7686864a9f93 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:42.295004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:42.295941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:42.296224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:42.296512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] PATH_INFO: `/v3/projects/8b116b8e4fca4ad794574e0e8f8eafb5/users/aadee117c3c04af39d031b35d96cf656/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:42.298542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b116b8e4fca4ad794574e0e8f8eafb5, user_id=aadee117c3c04af39d031b35d96cf656, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:42.301245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:42.324840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 979/1944] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:42 2022] PUT /identity/v3/projects/8b116b8e4fca4ad794574e0e8f8eafb5/users/aadee117c3c04af39d031b35d96cf656/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:42.335549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b877de-5a41-4661-93e7-589343e7d609 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:42.335836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b877de-5a41-4661-93e7-589343e7d609 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:42.336175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b877de-5a41-4661-93e7-589343e7d609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:42.375386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-93b877de-5a41-4661-93e7-589343e7d609 None None] MFA Rules not processed for user `aadee117c3c04af39d031b35d96cf656`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:42.376843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-93b877de-5a41-4661-93e7-589343e7d609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:42.491986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 966/1945] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:42.503128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:42.505258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:42.575705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf6cd1-9ff2-4447-b516-02ba0001ce9a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:42.580180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:42.581034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:42.581437 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:42.581898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:42.592381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:42.594098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:42.605256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 980/1946] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:43.057332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b877de-5a41-4661-93e7-589343e7d609 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:43.141305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b877de-5a41-4661-93e7-589343e7d609 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:43.148566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:43.150270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:43.150772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:43.151245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] PATH_INFO: `/v3/users/9dbc2b4ecfab411fafceb1fee0a6355d` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:43.153530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dbc2b4ecfab411fafceb1fee0a6355d)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:43.157476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:43.222931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:43.225688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:43.227480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dbc2b4ecfab411fafceb1fee0a6355d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.228176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dbc2b4ecfab411fafceb1fee0a6355d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.229003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dbc2b4ecfab411fafceb1fee0a6355d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.238422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dbc2b4ecfab411fafceb1fee0a6355d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.241223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a908f-30e4-4ef1-af2b-379f80bace3c service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:43.246819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:43.248111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:43.248652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:43.249359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] PATH_INFO: `/v3/users/71a10f7b280a42ffb7672a1c629b3200` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:43.251266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-07ed7740-aeda-460a-96bf-7fdef520c919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dbc2b4ecfab411fafceb1fee0a6355d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.251644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a10f7b280a42ffb7672a1c629b3200)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:43.254054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:43.267066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 967/1947] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:43 2022] DELETE /identity/v3/users/9dbc2b4ecfab411fafceb1fee0a6355d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:43.271901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:43.272330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:43.272666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:43.294281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:43.296723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:43.327817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a10f7b280a42ffb7672a1c629b3200'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.345098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a10f7b280a42ffb7672a1c629b3200'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.347617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a10f7b280a42ffb7672a1c629b3200'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.349023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a10f7b280a42ffb7672a1c629b3200'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.373699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-c4a0d48e-f8b2-40b0-a962-de10b8214351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a10f7b280a42ffb7672a1c629b3200'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:43.390924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 981/1948] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:43 2022] DELETE /identity/v3/users/71a10f7b280a42ffb7672a1c629b3200 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:53:43.395350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac2a8d-7088-4ca5-9d3c-91f5ea20291f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:43.395620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac2a8d-7088-4ca5-9d3c-91f5ea20291f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:43.395971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac2a8d-7088-4ca5-9d3c-91f5ea20291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:43.403874 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 968/1949] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:43.413165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:43.414772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:43.416929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-efac2a8d-7088-4ca5-9d3c-91f5ea20291f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:43.418174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-efac2a8d-7088-4ca5-9d3c-91f5ea20291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:43.501068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 982/1950] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:43.510488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac2a8d-7088-4ca5-9d3c-91f5ea20291f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:43.515061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190bdad-6784-464d-ba3d-f88046cd0aea None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:43.575608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:43.576333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:43.576774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:43.577097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:43.591499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:43.593494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:43.610293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 969/1951] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:53:43.636973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac2a8d-7088-4ca5-9d3c-91f5ea20291f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:43.641326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:43.642026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:43.642308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:43.642939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:43.656644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:43.658204 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:43.670442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 983/1952] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:53:44.175999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:44.184852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:44.215484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bdd2a-f4af-48d5-898b-b5854d01fa9e service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:44.224347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:44.225338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:44.225934 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:44.226475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] PATH_INFO: `/v3/projects/6e1e07f7318b4fb48bafdf7b62173533` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:44.228616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1e07f7318b4fb48bafdf7b62173533)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:44.230757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:44.293620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:44.296317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1e07f7318b4fb48bafdf7b62173533', 'request_id': 'req-7b54f21d-405a-4d0c-b05a-13f9e24539d3'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.296526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:44.304113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1e07f7318b4fb48bafdf7b62173533', 'request_id': 'req-7b54f21d-405a-4d0c-b05a-13f9e24539d3'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.311682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1e07f7318b4fb48bafdf7b62173533', 'request_id': 'req-7b54f21d-405a-4d0c-b05a-13f9e24539d3'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.325653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88b2a-97fb-499b-9197-9dc3ae295c24 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:44.337695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] The token cache is being invalidate because project 6e1e07f7318b4fb48bafdf7b62173533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:44.337996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.340449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.349005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 970/1953] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:44 2022] DELETE /identity/v3/projects/6e1e07f7318b4fb48bafdf7b62173533 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:44.359301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:44.363432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:44.453402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:44.455609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:44.455609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:44.455609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] PATH_INFO: `/v3/projects/08bf5349308b419d80b8d0076a7d467f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:44.456874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08bf5349308b419d80b8d0076a7d467f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:44.458410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:44.484302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54f21d-405a-4d0c-b05a-13f9e24539d3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:44.492197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:44.493192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:44.493471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:44.493796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] PATH_INFO: `/v3/users/d6834f8521044600a955554d1c2d6f33` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:44.495772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6834f8521044600a955554d1c2d6f33)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:44.497105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:44.533701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08bf5349308b419d80b8d0076a7d467f', 'request_id': 'req-d62032af-23fc-46ec-b711-fda9554b8f0f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.542943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08bf5349308b419d80b8d0076a7d467f', 'request_id': 'req-d62032af-23fc-46ec-b711-fda9554b8f0f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.551739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08bf5349308b419d80b8d0076a7d467f', 'request_id': 'req-d62032af-23fc-46ec-b711-fda9554b8f0f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.565281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] The token cache is being invalidate because project 08bf5349308b419d80b8d0076a7d467f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:44.565617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.566549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.568771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 984/1954] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:44 2022] DELETE /identity/v3/projects/08bf5349308b419d80b8d0076a7d467f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:53:44.572978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:44.575069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:44.576600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6834f8521044600a955554d1c2d6f33'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.577447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6834f8521044600a955554d1c2d6f33'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.578203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6834f8521044600a955554d1c2d6f33'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.589905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6834f8521044600a955554d1c2d6f33'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.608823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6834f8521044600a955554d1c2d6f33'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.632001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 971/1955] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:44 2022] DELETE /identity/v3/users/d6834f8521044600a955554d1c2d6f33 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:44.658980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62032af-23fc-46ec-b711-fda9554b8f0f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:44.725419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:44.726722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:44.727007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:44.727283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] PATH_INFO: `/v3/users/aadee117c3c04af39d031b35d96cf656` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:44.728953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadee117c3c04af39d031b35d96cf656)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:44.731212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:44.813585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadee117c3c04af39d031b35d96cf656'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.828977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadee117c3c04af39d031b35d96cf656'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.830818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadee117c3c04af39d031b35d96cf656'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.832457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadee117c3c04af39d031b35d96cf656'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.874749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d663c1b7-1d1a-4632-be01-cae8e266acda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadee117c3c04af39d031b35d96cf656'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:44.919805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 985/1956] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:44 2022] DELETE /identity/v3/users/aadee117c3c04af39d031b35d96cf656 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:45.085458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:45.087192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:45.122394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83637e-7cc6-4c96-948d-05a1d26be19c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:45.194536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:45.195674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:45.196022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:45.196644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] PATH_INFO: `/v3/projects/d4095820724c4470a4a5d655e192b502` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:45.198542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4095820724c4470a4a5d655e192b502)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:45.200854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:45.295010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4095820724c4470a4a5d655e192b502', 'request_id': 'req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:45.305873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4095820724c4470a4a5d655e192b502', 'request_id': 'req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:45.330177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4095820724c4470a4a5d655e192b502', 'request_id': 'req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:45.355036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] The token cache is being invalidate because project d4095820724c4470a4a5d655e192b502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:45.355564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:45.356708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c7a17bc5-dc58-4b83-b409-379a78bb0a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:45.363475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 972/1957] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:45 2022] DELETE /identity/v3/projects/d4095820724c4470a4a5d655e192b502 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:45.365152 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07f03-5233-434d-bf0d-de1b126b201d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:45.365654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07f03-5233-434d-bf0d-de1b126b201d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:45.365737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07f03-5233-434d-bf0d-de1b126b201d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:45.394916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a0a07f03-5233-434d-bf0d-de1b126b201d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:45.397545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a0a07f03-5233-434d-bf0d-de1b126b201d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:45.491582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8901a-5099-4ac6-ab5a-1be2aa7fbdfc None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:45.492176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8901a-5099-4ac6-ab5a-1be2aa7fbdfc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:45.492550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8901a-5099-4ac6-ab5a-1be2aa7fbdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:45.502245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 986/1958] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:45.688730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-acf8901a-5099-4ac6-ab5a-1be2aa7fbdfc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:45.690152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-acf8901a-5099-4ac6-ab5a-1be2aa7fbdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:45.707476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 973/1959] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:45.711511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a07f03-5233-434d-bf0d-de1b126b201d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:45.729328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a07f03-5233-434d-bf0d-de1b126b201d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:45.736725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:45.737589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:45.738856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:45.739271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:45.741201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:45.742443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:45.744409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:45.776381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 987/1960] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:45.781877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8901a-5099-4ac6-ab5a-1be2aa7fbdfc None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:45.799102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8901a-5099-4ac6-ab5a-1be2aa7fbdfc None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:45.806769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:45.808586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:45.819298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:45.820757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:45.822084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:45.824208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:45.825483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:45.834221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:45.850215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128126-6949-4ef3-a8b6-6e38a5f2377a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:45.874657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:45.876286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:45.876713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:45.879556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] PATH_INFO: `/v3/projects/8b116b8e4fca4ad794574e0e8f8eafb5` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:45.883290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b116b8e4fca4ad794574e0e8f8eafb5)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:45.885620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:45.921537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 974/1961] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:45 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:45.923850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:45.976896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b116b8e4fca4ad794574e0e8f8eafb5', 'request_id': 'req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:45.989332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b116b8e4fca4ad794574e0e8f8eafb5', 'request_id': 'req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.006776 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b116b8e4fca4ad794574e0e8f8eafb5', 'request_id': 'req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.028321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] The token cache is being invalidate because project 8b116b8e4fca4ad794574e0e8f8eafb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:46.028686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.029534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6033d52f-bb06-4ceb-8f74-56a5eb4d4826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.031572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 988/1962] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:45 2022] DELETE /identity/v3/projects/8b116b8e4fca4ad794574e0e8f8eafb5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:46.039666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fb4d6-421f-40d4-a4a1-71c59e0d4c9f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.040908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fb4d6-421f-40d4-a4a1-71c59e0d4c9f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.041565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fb4d6-421f-40d4-a4a1-71c59e0d4c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.059957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb597a2-0d11-4898-a9bb-9710df0bce41 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.073132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f73fb4d6-421f-40d4-a4a1-71c59e0d4c9f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:46.074525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f73fb4d6-421f-40d4-a4a1-71c59e0d4c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.136752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.137564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.137936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.138686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.139548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.141929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.165604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 975/1963] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:45 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:46.170347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.172594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.174572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 989/1964] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:46.182821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f345d4-f66e-4531-9c24-190dd3bcaef5 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.183379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f345d4-f66e-4531-9c24-190dd3bcaef5 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.183840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f345d4-f66e-4531-9c24-190dd3bcaef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.194625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c7ec-5e48-48f0-8bb1-f759190ad12e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.199929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.200970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.201315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.201592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.202436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.204037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.211463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-42f345d4-f66e-4531-9c24-190dd3bcaef5 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:46.212987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-42f345d4-f66e-4531-9c24-190dd3bcaef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.222213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 990/1965] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:46.227294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f345d4-f66e-4531-9c24-190dd3bcaef5 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.231877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 976/1966] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:46.233255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.236230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.242297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f345d4-f66e-4531-9c24-190dd3bcaef5 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.258427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.259094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.259414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.259694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.260449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:46.260758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.261879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8df85-c745-42d4-b398-db75feff0f3c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.266597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.271493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.273087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.273592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.274365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] PATH_INFO: `/v3/users/b4a70ea6369348bea63dc3eac7ac0380` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.277760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a70ea6369348bea63dc3eac7ac0380)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.280905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 991/1967] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:46.281661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.286431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.298122 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21436-62f1-4c56-8509-11086de45138 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.303950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.304640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.305017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.305662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.306821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.308856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.333097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 992/1968] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:46.335498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.349931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e2709-950d-43da-827f-a7f45bf05237 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.355394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.356174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.356512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.356840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.357653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.359916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.368480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a70ea6369348bea63dc3eac7ac0380'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.369310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a70ea6369348bea63dc3eac7ac0380'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.369985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a70ea6369348bea63dc3eac7ac0380'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.382990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a70ea6369348bea63dc3eac7ac0380'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.395762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 993/1969] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:46.395762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.395762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.413085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a70ea6369348bea63dc3eac7ac0380'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.439668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 977/1970] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:46 2022] DELETE /identity/v3/users/b4a70ea6369348bea63dc3eac7ac0380 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:46.439668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.440930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.482107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878475ac-dbe2-4fbf-ac1e-f7784b1073e5 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.517325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca2bb7-277a-43e9-8237-7856a5801003 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.522741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.523442 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.523725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.523997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.524773 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.527353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.551695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.552851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.553022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.553568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] PATH_INFO: `/v3/projects/c4ed736b867c49ba9321ca3451550f9d/users/df2fe995e416476e90e697b29fad7f96/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.554621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 978/1971] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:46.559787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.572175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9945db-be5f-449f-9382-85323e31bc34 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.576980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ed736b867c49ba9321ca3451550f9d, user_id=df2fe995e416476e90e697b29fad7f96, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.579266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe4a9-2228-4b25-a08f-ea1306b6f22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.580170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.580402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe4a9-2228-4b25-a08f-ea1306b6f22a admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.580776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe4a9-2228-4b25-a08f-ea1306b6f22a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.581267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe4a9-2228-4b25-a08f-ea1306b6f22a admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.582356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe4a9-2228-4b25-a08f-ea1306b6f22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.584238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe4a9-2228-4b25-a08f-ea1306b6f22a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.603574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 994/1972] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:46 2022] PUT /identity/v3/projects/c4ed736b867c49ba9321ca3451550f9d/users/df2fe995e416476e90e697b29fad7f96/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:46.609864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 979/1973] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:46.614294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.621150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb79cc4-258c-47fa-b80c-8e1edab04bdf None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.621526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb79cc4-258c-47fa-b80c-8e1edab04bdf None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.621875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb79cc4-258c-47fa-b80c-8e1edab04bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.658309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ebb79cc4-258c-47fa-b80c-8e1edab04bdf None None] MFA Rules not processed for user `df2fe995e416476e90e697b29fad7f96`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:46.660539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ebb79cc4-258c-47fa-b80c-8e1edab04bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.722365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4ed1-28a0-4b6f-876a-2808cfd38b9a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.727223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.727909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.728188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.728463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.729280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.730901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.744713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 995/1974] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:46.744713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.744713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.750698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 980/1975] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:46.754348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb79cc4-258c-47fa-b80c-8e1edab04bdf None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.757797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea5ae2-1383-4401-895a-1add2cdc3d5c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.762391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.762965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb79cc4-258c-47fa-b80c-8e1edab04bdf None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.763655 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.763964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.764235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] PATH_INFO: `/v3/projects/958a8323b6914605b4767698e1510825` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.765461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] RBAC: Authorizing `identity:delete_project(project_id=958a8323b6914605b4767698e1510825)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.767373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.768358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.769272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.769544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.769811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] PATH_INFO: `/v3/projects/e09732461cb94ed590e8336509f571ab/users/98a6b33672544ac797c04e5ccd005354/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.789466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09732461cb94ed590e8336509f571ab, user_id=98a6b33672544ac797c04e5ccd005354, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.795030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:46.811144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 981/1976] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:46 2022] PUT /identity/v3/projects/e09732461cb94ed590e8336509f571ab/users/98a6b33672544ac797c04e5ccd005354/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:46.812160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:46.820374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958a8323b6914605b4767698e1510825', 'request_id': 'req-be9ce006-0d6f-454a-8c24-91ab32b1138d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.830612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958a8323b6914605b4767698e1510825', 'request_id': 'req-be9ce006-0d6f-454a-8c24-91ab32b1138d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.836822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958a8323b6914605b4767698e1510825', 'request_id': 'req-be9ce006-0d6f-454a-8c24-91ab32b1138d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.849306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] The token cache is being invalidate because project 958a8323b6914605b4767698e1510825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:46.849723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.850479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-be9ce006-0d6f-454a-8c24-91ab32b1138d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:46.852356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 996/1977] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:46 2022] DELETE /identity/v3/projects/958a8323b6914605b4767698e1510825 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:46.853454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ace0cb-5eb7-4203-8e3c-6c55e98b0064 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.853744 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ace0cb-5eb7-4203-8e3c-6c55e98b0064 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.854048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ace0cb-5eb7-4203-8e3c-6c55e98b0064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.882044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-30ace0cb-5eb7-4203-8e3c-6c55e98b0064 None None] MFA Rules not processed for user `98a6b33672544ac797c04e5ccd005354`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:46.883304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-30ace0cb-5eb7-4203-8e3c-6c55e98b0064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:46.888949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02b790-f12d-42af-93dd-1fb3801cab9e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:46.962611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:46.963431 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.963794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.964163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.964710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 997/1978] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:46.965412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:46.966143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:46.966491 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:46.966798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:46.969163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.000525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:47.001995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.002564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 982/1979] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:47.003677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.005517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.108768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 998/1980] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:47.108768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.109759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b242f-f668-4c80-94c8-7665a090d63c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.109856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.114114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.114991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.115281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.115571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.120840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c46fb8-09be-4440-b3e9-a962e5768139 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.129499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c1c84-5a42-4fc3-8615-3d3739c87afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.130472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c1c84-5a42-4fc3-8615-3d3739c87afc admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.130914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c1c84-5a42-4fc3-8615-3d3739c87afc admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.131208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c1c84-5a42-4fc3-8615-3d3739c87afc admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.132076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c1c84-5a42-4fc3-8615-3d3739c87afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.134170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c1c84-5a42-4fc3-8615-3d3739c87afc admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.160846 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 999/1981] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:47 2022] POST /identity/v3/users => generated 298 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:47.162393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c63f6-75b5-4495-9a5f-fbd91f2dc211 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.163193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c63f6-75b5-4495-9a5f-fbd91f2dc211 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.163644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c63f6-75b5-4495-9a5f-fbd91f2dc211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.184492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-979c63f6-75b5-4495-9a5f-fbd91f2dc211 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:47.185886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-979c63f6-75b5-4495-9a5f-fbd91f2dc211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.188525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.190109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.195730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1000/1982] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:47.196632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c63f6-75b5-4495-9a5f-fbd91f2dc211 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.201313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 983/1983] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:47.202430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.207773 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c63f6-75b5-4495-9a5f-fbd91f2dc211 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.214781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.215481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e314d-413b-4b4a-9ddf-867692f99b6d service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.215605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.215964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.216884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.217562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.221158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.222228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.223050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.223541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.224032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.225170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:47.225549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.227255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.237963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1001/1984] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:47.239674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.241998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.243192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 984/1985] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:47.244266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.255251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7bc01-479f-4c68-b3ad-c31d0639cf1a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.256444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5ae93-780f-41ca-8992-aa1ca283836f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.261663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.262749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.263049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.263548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.263762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] PATH_INFO: `/v3/projects/24009a4d4cab47d4933a16755b42b4bb/users/66d9375005a8489aa97fa7bd118eec53/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.264394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.264919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.265258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] PATH_INFO: `/v3/users/98a6b33672544ac797c04e5ccd005354` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.266731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a6b33672544ac797c04e5ccd005354)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.268741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.282284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24009a4d4cab47d4933a16755b42b4bb, user_id=66d9375005a8489aa97fa7bd118eec53, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.289161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.306556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 985/1986] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:47 2022] PUT /identity/v3/projects/24009a4d4cab47d4933a16755b42b4bb/users/66d9375005a8489aa97fa7bd118eec53/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:47.307230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.343108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a6b33672544ac797c04e5ccd005354'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:47.357854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a6b33672544ac797c04e5ccd005354'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:47.358722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a6b33672544ac797c04e5ccd005354'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:47.359426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a6b33672544ac797c04e5ccd005354'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:47.368500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a6b33672544ac797c04e5ccd005354'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:47.386819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1002/1987] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:47 2022] DELETE /identity/v3/users/98a6b33672544ac797c04e5ccd005354 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:47.386819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.386819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.395959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf7614-7219-4400-83b7-69edbd3a8975 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.466911 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88721334-805c-4068-95ec-e23029fc6ee3 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.468002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.468888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.469217 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.469748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.470604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.472900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.473053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d226d9f-711f-477b-85e8-332c0d717b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.473817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d226d9f-711f-477b-85e8-332c0d717b81 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.474160 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d226d9f-711f-477b-85e8-332c0d717b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.474495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d226d9f-711f-477b-85e8-332c0d717b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.475358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d226d9f-711f-477b-85e8-332c0d717b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.477102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d226d9f-711f-477b-85e8-332c0d717b81 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.490138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1003/1988] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:47.491601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ceb53-142b-4ea4-95de-e42d904e86e7 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.492003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ceb53-142b-4ea4-95de-e42d904e86e7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.492380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ceb53-142b-4ea4-95de-e42d904e86e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.514451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 986/1989] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:47 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:47.515816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.547669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-de4ceb53-142b-4ea4-95de-e42d904e86e7 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:47.548972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-de4ceb53-142b-4ea4-95de-e42d904e86e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.608281 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c206890-0323-415e-8a79-0fbe10b94905 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.620745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.621700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.622083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.622422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] PATH_INFO: `/v3/projects/24009a4d4cab47d4933a16755b42b4bb/users/66d9375005a8489aa97fa7bd118eec53/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.626066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24009a4d4cab47d4933a16755b42b4bb, user_id=66d9375005a8489aa97fa7bd118eec53, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.631057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.648266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 987/1990] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:47 2022] PUT /identity/v3/projects/24009a4d4cab47d4933a16755b42b4bb/users/66d9375005a8489aa97fa7bd118eec53/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:47.653224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.653224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.670629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1004/1991] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:47.672358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845896d-ff64-48fc-90d0-cc97c0d4bbde None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.672891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845896d-ff64-48fc-90d0-cc97c0d4bbde None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.673405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845896d-ff64-48fc-90d0-cc97c0d4bbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.705812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-5845896d-ff64-48fc-90d0-cc97c0d4bbde None None] MFA Rules not processed for user `66d9375005a8489aa97fa7bd118eec53`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:47.707235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5845896d-ff64-48fc-90d0-cc97c0d4bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.732902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f327ca-abe1-494b-a9e4-9579291bf4f7 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.737808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.738501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.738783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.739065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.739810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.741814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.764581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 988/1992] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:47 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:47.765541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.767480 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.801359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1005/1993] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:47.802233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845896d-ff64-48fc-90d0-cc97c0d4bbde None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.817485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845896d-ff64-48fc-90d0-cc97c0d4bbde None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.822276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.822951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.823226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.823486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.824255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.826275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.835443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c853c38-98a9-431f-8beb-531d3584d3c0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.837318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1006/1994] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:47.838369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.842820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.843582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.844316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.844627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.844943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.857194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.858679 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.870032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 989/1995] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:47.871073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.884376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031a1b8-a0d7-429e-9410-87b260d9b0b7 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.890992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.891685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.892359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.892652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.893441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.895075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.907200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 990/1996] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:47.911407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:47.921862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e994db5-9f7b-4f5a-83e9-c77b345c6972 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.925764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.926443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.926677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf47e-b5b8-44ed-8bb8-c6e832b2738e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:47.926813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.926986 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.929305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:47.937471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e9598-41eb-4611-997e-c30657801b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:47.939310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e9598-41eb-4611-997e-c30657801b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:47.939909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e9598-41eb-4611-997e-c30657801b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:47.940702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e9598-41eb-4611-997e-c30657801b75 admin admin] PATH_INFO: `/v3/projects/918957f1856d4d7083f376c1946d9524/users/e81a0ee97abc4d17b3449d84af31bbb8/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:47.970955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e9598-41eb-4611-997e-c30657801b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918957f1856d4d7083f376c1946d9524, user_id=e81a0ee97abc4d17b3449d84af31bbb8, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:47.974225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e9598-41eb-4611-997e-c30657801b75 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:47.996571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 991/1997] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:47 2022] PUT /identity/v3/projects/918957f1856d4d7083f376c1946d9524/users/e81a0ee97abc4d17b3449d84af31bbb8/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:48.008387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.009296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.009897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.025664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:48.027778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:48.043529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1007/1998] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:48.062353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] MFA Rules not processed for user `e81a0ee97abc4d17b3449d84af31bbb8`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:48.064350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:48.161576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 992/1999] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:48.174496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:48.177117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:48.239212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:48.240741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:48.319365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd732612-b93b-433e-9282-e50f56404ef7 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:48.375335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188532d9-ba3b-4cb1-b781-d98124c1b945 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:48.383100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:48.384784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.385392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.385966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] PATH_INFO: `/v3/projects/e09732461cb94ed590e8336509f571ab` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.389001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09732461cb94ed590e8336509f571ab)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:48.393279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:48.442549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b5bbd-fa1d-43f3-aaec-80a5b2ad2a91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:48.443675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bbd-fa1d-43f3-aaec-80a5b2ad2a91 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.444178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bbd-fa1d-43f3-aaec-80a5b2ad2a91 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.444653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bbd-fa1d-43f3-aaec-80a5b2ad2a91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.465084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09732461cb94ed590e8336509f571ab', 'request_id': 'req-06d72731-b048-44af-b7b6-d9ebc4897a59'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:48.466017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5bbd-fa1d-43f3-aaec-80a5b2ad2a91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:48.470781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5bbd-fa1d-43f3-aaec-80a5b2ad2a91 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:48.472628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09732461cb94ed590e8336509f571ab', 'request_id': 'req-06d72731-b048-44af-b7b6-d9ebc4897a59'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:48.480643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09732461cb94ed590e8336509f571ab', 'request_id': 'req-06d72731-b048-44af-b7b6-d9ebc4897a59'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:48.482406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1008/2000] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:48.497303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] The token cache is being invalidate because project e09732461cb94ed590e8336509f571ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:48.497744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:48.498806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-06d72731-b048-44af-b7b6-d9ebc4897a59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:48.500442 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 993/2001] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:48 2022] DELETE /identity/v3/projects/e09732461cb94ed590e8336509f571ab => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:53:48.506205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c567b-7c6c-473a-99ca-3cc5c3279c30 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.506867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c567b-7c6c-473a-99ca-3cc5c3279c30 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.507117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c567b-7c6c-473a-99ca-3cc5c3279c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.549371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-114c567b-7c6c-473a-99ca-3cc5c3279c30 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:48.552570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-114c567b-7c6c-473a-99ca-3cc5c3279c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:48.650861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1009/2002] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:48.660296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d008a-5956-4b1d-825a-3beeabed4153 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.660713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d008a-5956-4b1d-825a-3beeabed4153 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.661296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d008a-5956-4b1d-825a-3beeabed4153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.681672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d40d008a-5956-4b1d-825a-3beeabed4153 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:48.682905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d40d008a-5956-4b1d-825a-3beeabed4153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:48.691321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 994/2003] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:48.696466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c567b-7c6c-473a-99ca-3cc5c3279c30 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:48.710506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c567b-7c6c-473a-99ca-3cc5c3279c30 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:48.718644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:48.719648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.720483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.720978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.722331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:48.722891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:48.725831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:48.746665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1010/2004] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:48.752339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d008a-5956-4b1d-825a-3beeabed4153 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:48.763793 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d008a-5956-4b1d-825a-3beeabed4153 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:48.771020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:48.773040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.773383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.773880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.775123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:48.778801 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:48.825170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 995/2005] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:48 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:48.830174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:48.841539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ad732-ae9d-4079-8393-18ed99d99f78 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:48.932733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:48.933488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:48.933770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:48.934065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:48.934770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:48.937005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:48.968258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1011/2006] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:48 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:53:48.974078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:48.998239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94dc45c-8717-434d-948c-d265ac14e1a3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:49.005998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:49.006983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.007472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.007975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.009049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:49.011575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:49.024928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 996/2007] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:49.037104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:49.051336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9addb-4af7-46b2-8336-9cf61869a624 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:49.057086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecb758-6e98-4845-93f5-fc8aee9123f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:49.057785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecb758-6e98-4845-93f5-fc8aee9123f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.058073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecb758-6e98-4845-93f5-fc8aee9123f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.058464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecb758-6e98-4845-93f5-fc8aee9123f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.059090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecb758-6e98-4845-93f5-fc8aee9123f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:49.060837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecb758-6e98-4845-93f5-fc8aee9123f4 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:49.075998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1012/2008] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:49.081631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:49.093161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95328e95-3b74-4643-8dd2-2824a0dd96fe admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:49.100520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:49.101918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.102375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.102818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] PATH_INFO: `/v3/projects/110a9fb9ab9e438e81f887e55388a024/users/269b6be4e23340699583b55434d3f63f/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.121062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=110a9fb9ab9e438e81f887e55388a024, user_id=269b6be4e23340699583b55434d3f63f, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:49.124550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:49.141960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 997/2009] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:49 2022] PUT /identity/v3/projects/110a9fb9ab9e438e81f887e55388a024/users/269b6be4e23340699583b55434d3f63f/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:53:49.152745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68f2ef-0dbd-4751-a6d3-359dab4a5b1c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.153106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68f2ef-0dbd-4751-a6d3-359dab4a5b1c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.153428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68f2ef-0dbd-4751-a6d3-359dab4a5b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.185883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1f68f2ef-0dbd-4751-a6d3-359dab4a5b1c None None] MFA Rules not processed for user `269b6be4e23340699583b55434d3f63f`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:49.187176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1f68f2ef-0dbd-4751-a6d3-359dab4a5b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:49.260493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1013/2010] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:49.272599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:49.274456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:49.344416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93274b0-4425-4a0f-b2cb-6a791026a47c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:49.348241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6013724-c857-4284-b63c-0ac3126641d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:49.348941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6013724-c857-4284-b63c-0ac3126641d0 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.349225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6013724-c857-4284-b63c-0ac3126641d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.349515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6013724-c857-4284-b63c-0ac3126641d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.359213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6013724-c857-4284-b63c-0ac3126641d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:49.360998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6013724-c857-4284-b63c-0ac3126641d0 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:49.371120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 998/2011] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:49.391769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68f2ef-0dbd-4751-a6d3-359dab4a5b1c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:49.457456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68f2ef-0dbd-4751-a6d3-359dab4a5b1c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:49.462760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:49.463661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.463918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.464216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] PATH_INFO: `/v3/users/269b6be4e23340699583b55434d3f63f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.465564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=269b6be4e23340699583b55434d3f63f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:49.467612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:49.519478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269b6be4e23340699583b55434d3f63f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:49.532276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269b6be4e23340699583b55434d3f63f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:49.533052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269b6be4e23340699583b55434d3f63f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:49.533658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269b6be4e23340699583b55434d3f63f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:49.542287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269b6be4e23340699583b55434d3f63f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:49.558935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1014/2012] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:53:49 2022] DELETE /identity/v3/users/269b6be4e23340699583b55434d3f63f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:49.563822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.564087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.564379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.582987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:49.584237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:49.652274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 999/2013] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:49.663650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:49.665555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:49.745331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2485e38-b9ee-486d-b435-cb5234477ddb admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:49.749104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5df5-cf2e-46a4-bf58-7136179d9971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:49.749775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5df5-cf2e-46a4-bf58-7136179d9971 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:49.750057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5df5-cf2e-46a4-bf58-7136179d9971 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:49.750329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5df5-cf2e-46a4-bf58-7136179d9971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:49.760180 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f5df5-cf2e-46a4-bf58-7136179d9971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:49.761671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f5df5-cf2e-46a4-bf58-7136179d9971 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:49.772630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1015/2014] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:50.106476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.108023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:50.121504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b958d05-bcd7-4182-8ca4-04c7dd5f9218 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.127299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.128319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.128691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.129227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] PATH_INFO: `/v3/projects/110a9fb9ab9e438e81f887e55388a024` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.130691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] RBAC: Authorizing `identity:delete_project(project_id=110a9fb9ab9e438e81f887e55388a024)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.132922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.192132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110a9fb9ab9e438e81f887e55388a024', 'request_id': 'req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:50.199891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110a9fb9ab9e438e81f887e55388a024', 'request_id': 'req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:50.206088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110a9fb9ab9e438e81f887e55388a024', 'request_id': 'req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:50.220400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] The token cache is being invalidate because project 110a9fb9ab9e438e81f887e55388a024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:53:50.220691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:50.221500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-63ec406a-a1a6-4841-8543-0c6c2dbdc98d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:53:50.222990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1000/2015] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:53:50 2022] DELETE /identity/v3/projects/110a9fb9ab9e438e81f887e55388a024 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:50.229306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f431-d85c-4e67-b02e-26613d443acb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.229816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f431-d85c-4e67-b02e-26613d443acb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.230349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5f431-d85c-4e67-b02e-26613d443acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.252550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-5bd5f431-d85c-4e67-b02e-26613d443acb None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:50.253843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5bd5f431-d85c-4e67-b02e-26613d443acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:50.320943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1016/2016] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:50.329553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6e89-4d83-4fb9-a153-2a1a30b9620a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.330443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6e89-4d83-4fb9-a153-2a1a30b9620a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.330443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6e89-4d83-4fb9-a153-2a1a30b9620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.352562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-81df6e89-4d83-4fb9-a153-2a1a30b9620a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:50.354032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-81df6e89-4d83-4fb9-a153-2a1a30b9620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:50.363409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1001/2017] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:50.368309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f431-d85c-4e67-b02e-26613d443acb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.378896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5f431-d85c-4e67-b02e-26613d443acb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.383923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.384624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.384947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.385220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.385984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:53:50.386284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.387959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.402414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1017/2018] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:53:50.409344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df6e89-4d83-4fb9-a153-2a1a30b9620a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.420446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df6e89-4d83-4fb9-a153-2a1a30b9620a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.426398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.427128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.427420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.427687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.428407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.430456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.460891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1002/2019] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:53:50 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:50.465475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.476452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fae17b-cf06-408b-9c59-f4030a5906f8 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.542332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.543077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.543293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.543589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.544319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.546489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.569611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1018/2020] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:53:50 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:53:50.574256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.585530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be94a0-ccfc-4d26-82e0-662513f99338 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.591375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.592113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.592392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.592659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.593462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.595158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.607631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1003/2021] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:53:50.612051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.622775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d31065-285b-4531-9972-f3ed9dd8c42c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.628118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186731f1-b7aa-43a6-b8c0-6ca60b51f4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.628894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186731f1-b7aa-43a6-b8c0-6ca60b51f4dc admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.629168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186731f1-b7aa-43a6-b8c0-6ca60b51f4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.629448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186731f1-b7aa-43a6-b8c0-6ca60b51f4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.630219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186731f1-b7aa-43a6-b8c0-6ca60b51f4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.631869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186731f1-b7aa-43a6-b8c0-6ca60b51f4dc admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.644654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1019/2022] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:53:50.649127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.660080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f506-86d8-4e4a-9b6f-1bff0bc4e49a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.665095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.666029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.666310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.666584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] PATH_INFO: `/v3/projects/f2a9a0c14546490db1746dda5aedd3ea/users/7533ae6ea6774f7eb853455d086f939d/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.684950 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a9a0c14546490db1746dda5aedd3ea, user_id=7533ae6ea6774f7eb853455d086f939d, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.687760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.704619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1004/2023] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:53:50 2022] PUT /identity/v3/projects/f2a9a0c14546490db1746dda5aedd3ea/users/7533ae6ea6774f7eb853455d086f939d/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:53:50.710158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476f871-0bd5-4159-96a1-90ff6b914962 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.710441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476f871-0bd5-4159-96a1-90ff6b914962 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.710711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476f871-0bd5-4159-96a1-90ff6b914962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.730775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a476f871-0bd5-4159-96a1-90ff6b914962 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:50.732082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a476f871-0bd5-4159-96a1-90ff6b914962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:50.803445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1020/2024] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:50.814351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:50.816416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:50.876222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd8bf6-904c-47c5-a80f-da3c791a9d3a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:50.879894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:50.880539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:50.880842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:50.881119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:50.891058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:50.892507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:50.904377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1005/2025] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:53:51.626062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5810-5580-40a4-9ec0-224a168eec7a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:51.626920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5810-5580-40a4-9ec0-224a168eec7a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:51.626920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5810-5580-40a4-9ec0-224a168eec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:51.656862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-14ba5810-5580-40a4-9ec0-224a168eec7a None None] MFA Rules not processed for user `7533ae6ea6774f7eb853455d086f939d`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:51.658159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-14ba5810-5580-40a4-9ec0-224a168eec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:51.750355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1021/2026] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:53:51.761452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:51.762978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:51.846090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd32674-d2fb-4840-9519-7c59d766074f service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:51.849608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:51.850282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:51.850543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:51.850816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:51.860601 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:51.862048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:51.871425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1006/2027] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:53:56.635039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:56.635577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:56.635577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:56.655071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] MFA Rules not processed for user `7533ae6ea6774f7eb853455d086f939d`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:53:56.656514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:56.675953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1022/2028] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:53:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:53:57.776354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:53:57.787606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13575a-3122-406f-80ff-92000f3bae8d service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:53:57.792719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a83ff-20bd-4db2-a062-2f21a7896dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:53:57.793578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a83ff-20bd-4db2-a062-2f21a7896dab service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:53:57.793907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a83ff-20bd-4db2-a062-2f21a7896dab service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:53:57.794061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a83ff-20bd-4db2-a062-2f21a7896dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:53:57.796321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f57a83ff-20bd-4db2-a062-2f21a7896dab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:53:57.874259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a83ff-20bd-4db2-a062-2f21a7896dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:53:57.875995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57a83ff-20bd-4db2-a062-2f21a7896dab service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:53:57.890985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1007/2029] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:53:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:01.465563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:01.467469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:01.483337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33655ab1-468d-43b0-bb0d-ad859a3ac900 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:01.489050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:01.490066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:01.490410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:01.490806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] PATH_INFO: `/v3/users/e81a0ee97abc4d17b3449d84af31bbb8` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:01.492123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81a0ee97abc4d17b3449d84af31bbb8)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:01.494340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:01.557362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81a0ee97abc4d17b3449d84af31bbb8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:01.572729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81a0ee97abc4d17b3449d84af31bbb8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:01.574871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81a0ee97abc4d17b3449d84af31bbb8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:01.575579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81a0ee97abc4d17b3449d84af31bbb8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:01.584489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81a0ee97abc4d17b3449d84af31bbb8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:01.601057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1023/2030] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:01 2022] DELETE /identity/v3/users/e81a0ee97abc4d17b3449d84af31bbb8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:01.606206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:01.611625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:01.612000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:01.633347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:01.636047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:01.701164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1008/2031] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:01.714260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:01.721742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:01.792638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b094a-cdc7-47c5-967a-95784879ee1c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:01.797017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6c65f-7715-465c-80d3-b1e42441caac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:01.797726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c65f-7715-465c-80d3-b1e42441caac service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:01.798005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c65f-7715-465c-80d3-b1e42441caac service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:01.798523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6c65f-7715-465c-80d3-b1e42441caac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:01.809539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6c65f-7715-465c-80d3-b1e42441caac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:01.811416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6c65f-7715-465c-80d3-b1e42441caac service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:01.823149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1024/2032] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:02.222717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:02.224464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:02.242427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36528d-4fc5-4e37-8b37-e23711fdd34d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:02.252918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:02.254288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.254723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.255147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] PATH_INFO: `/v3/projects/918957f1856d4d7083f376c1946d9524` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.257666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=918957f1856d4d7083f376c1946d9524)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:02.260438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:02.349119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918957f1856d4d7083f376c1946d9524', 'request_id': 'req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:02.361593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918957f1856d4d7083f376c1946d9524', 'request_id': 'req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:02.373144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918957f1856d4d7083f376c1946d9524', 'request_id': 'req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:02.401654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] The token cache is being invalidate because project 918957f1856d4d7083f376c1946d9524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:02.402486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:02.404235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-5c8bf70e-034a-41f0-a595-b0dcc996fa1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:02.407938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1009/2033] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:02 2022] DELETE /identity/v3/projects/918957f1856d4d7083f376c1946d9524 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:02.416055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723a68-54ac-42b7-9d29-fa00a8cdb6ee None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.416737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723a68-54ac-42b7-9d29-fa00a8cdb6ee None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.419369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723a68-54ac-42b7-9d29-fa00a8cdb6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.449475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e1723a68-54ac-42b7-9d29-fa00a8cdb6ee None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:02.451112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e1723a68-54ac-42b7-9d29-fa00a8cdb6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:02.541531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1025/2034] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:02.555280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804ba62-8c34-4673-a76b-29d1555d18a7 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.555859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804ba62-8c34-4673-a76b-29d1555d18a7 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.556287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804ba62-8c34-4673-a76b-29d1555d18a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.587749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-f804ba62-8c34-4673-a76b-29d1555d18a7 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:02.589013 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f804ba62-8c34-4673-a76b-29d1555d18a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:02.599167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1010/2035] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:02.603487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1723a68-54ac-42b7-9d29-fa00a8cdb6ee None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:02.614482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1723a68-54ac-42b7-9d29-fa00a8cdb6ee None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:02.620497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:02.621406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.621995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.622403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.623173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:02.623485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:02.625199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:02.644115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1026/2036] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:54:02.645292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804ba62-8c34-4673-a76b-29d1555d18a7 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:02.655870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804ba62-8c34-4673-a76b-29d1555d18a7 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:02.661846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:02.662732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.663341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.663740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.664615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:02.668571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:02.712022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1011/2037] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:02 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:02.712531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:02.724964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf150f-4d69-40fb-85ef-37ee524fb29e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:02.812403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:02.813186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.813534 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.813862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.814640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:02.817019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:02.841512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1027/2038] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:02 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:02.842952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:02.853395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0056c9-fbf0-4581-96e6-52984f4477e8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:02.858590 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:02.859260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.859534 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.859811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.860556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:02.862222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:02.873990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1012/2039] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:02.877797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:02.889853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0beb5e-93ec-41e6-ba97-b16d76250b8e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:02.898517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e6312-bf67-4af1-8cbb-432e78c7f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:02.899217 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e6312-bf67-4af1-8cbb-432e78c7f34f admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.899506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e6312-bf67-4af1-8cbb-432e78c7f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.899791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e6312-bf67-4af1-8cbb-432e78c7f34f admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.900776 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e6312-bf67-4af1-8cbb-432e78c7f34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:02.902877 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e6312-bf67-4af1-8cbb-432e78c7f34f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:02.918079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1028/2040] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:02.922527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:02.933441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6ac17-8545-419a-9a8e-46e8cfc7015d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:02.938510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:02.940175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.940449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.940728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] PATH_INFO: `/v3/projects/60b5344da0a24c9bb99773903c762f14/users/2b4b2d6563804db8a1043f1fd2951001/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:02.959418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b5344da0a24c9bb99773903c762f14, user_id=2b4b2d6563804db8a1043f1fd2951001, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:02.962391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:02.978886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1013/2041] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:02 2022] PUT /identity/v3/projects/60b5344da0a24c9bb99773903c762f14/users/2b4b2d6563804db8a1043f1fd2951001/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:02.990970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:02.991396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:02.991766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:03.035486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] MFA Rules not processed for user `2b4b2d6563804db8a1043f1fd2951001`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:03.036947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:03.116847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1029/2042] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:03.129167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:03.131017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:03.205372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af1a44-d1d1-4970-8bf6-309d99795543 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:03.209405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832107-612a-4514-8cd4-57f336a9c0f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:03.210083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832107-612a-4514-8cd4-57f336a9c0f9 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:03.210364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832107-612a-4514-8cd4-57f336a9c0f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:03.210637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832107-612a-4514-8cd4-57f336a9c0f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:03.221207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832107-612a-4514-8cd4-57f336a9c0f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:03.223111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832107-612a-4514-8cd4-57f336a9c0f9 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:03.234398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1014/2043] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:04.972948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:04.974691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.091313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe550d-f39e-473e-9ac3-837f514f1cc3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:05.106621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:05.107578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:05.107857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:05.108132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] PATH_INFO: `/v3/users/df2fe995e416476e90e697b29fad7f96` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:05.109602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df2fe995e416476e90e697b29fad7f96)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:05.111739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:05.167518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df2fe995e416476e90e697b29fad7f96'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.181168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df2fe995e416476e90e697b29fad7f96'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.181907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df2fe995e416476e90e697b29fad7f96'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.182545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df2fe995e416476e90e697b29fad7f96'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.192280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df2fe995e416476e90e697b29fad7f96'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.205672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1030/2044] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:04 2022] DELETE /identity/v3/users/df2fe995e416476e90e697b29fad7f96 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:05.210787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:05.211787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:05.211787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:05.230703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:05.231943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.309566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1015/2045] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:05.319326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:05.320913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.395210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a4f6e-2863-4fbe-aee2-84bcc930a614 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:05.400087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:05.400971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:05.401354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:05.401617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:05.413125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:05.414585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:05.426596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1031/2046] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:05.679850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:05.681547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.697017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b89ad4-088f-4569-8faf-cb0b58da49c2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:05.708348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:05.709570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:05.709917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:05.710257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] PATH_INFO: `/v3/users/ddff5d5e6a5643529cae5f6cb075bd9e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:05.711824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddff5d5e6a5643529cae5f6cb075bd9e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:05.714799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:05.724103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:05.725663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.747997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0d024-e99f-4511-a94a-40c5efb75dc7 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:05.756600 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:05.758008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:05.758300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:05.758496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] PATH_INFO: `/v3/projects/c4ed736b867c49ba9321ca3451550f9d` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:05.761949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ed736b867c49ba9321ca3451550f9d)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:05.765253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:05.789332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddff5d5e6a5643529cae5f6cb075bd9e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.790224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddff5d5e6a5643529cae5f6cb075bd9e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.791155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddff5d5e6a5643529cae5f6cb075bd9e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.801846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddff5d5e6a5643529cae5f6cb075bd9e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.816658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b6fe5cd2-2b47-4483-899f-7c09f62b52be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddff5d5e6a5643529cae5f6cb075bd9e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.834654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1016/2047] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:05 2022] DELETE /identity/v3/users/ddff5d5e6a5643529cae5f6cb075bd9e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:05.836304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ed736b867c49ba9321ca3451550f9d', 'request_id': 'req-0d565b43-442a-48e4-a09a-dfb59bd532c0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.839827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:05.840143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:05.840526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:05.847884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ed736b867c49ba9321ca3451550f9d', 'request_id': 'req-0d565b43-442a-48e4-a09a-dfb59bd532c0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.856755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ed736b867c49ba9321ca3451550f9d', 'request_id': 'req-0d565b43-442a-48e4-a09a-dfb59bd532c0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.868993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:05.870417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.871158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] The token cache is being invalidate because project c4ed736b867c49ba9321ca3451550f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:05.871596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.872417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:05.874147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1032/2048] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:05 2022] DELETE /identity/v3/projects/c4ed736b867c49ba9321ca3451550f9d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:05.878282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:05.879781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.951453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1017/2049] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:05.951713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d565b43-442a-48e4-a09a-dfb59bd532c0 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:05.958101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:05.959049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:05.959327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:05.959595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] PATH_INFO: `/v3/users/66d9375005a8489aa97fa7bd118eec53` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:05.960873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d9375005a8489aa97fa7bd118eec53)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:05.963320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:05.964896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:05.965794 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.036464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d9375005a8489aa97fa7bd118eec53'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.037230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f422be-ad8f-4735-872f-95441cd8e388 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.040422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.041117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.041402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.041687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.048869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d9375005a8489aa97fa7bd118eec53'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.049601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d9375005a8489aa97fa7bd118eec53'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.050253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d9375005a8489aa97fa7bd118eec53'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.051207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.052698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.058433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d9375005a8489aa97fa7bd118eec53'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.063663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1018/2050] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:06.072407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1033/2051] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:05 2022] DELETE /identity/v3/users/66d9375005a8489aa97fa7bd118eec53 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:06.368721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.370978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:06.409939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84296fad-830f-4b15-8e92-3186330ae92a service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.421078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.423067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:06.515360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789917-e438-403e-a431-f61a1e712da5 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.521291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.522337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.522682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.523023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] PATH_INFO: `/v3/projects/24009a4d4cab47d4933a16755b42b4bb` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.524392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24009a4d4cab47d4933a16755b42b4bb)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.526672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.527322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.528494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.528784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.529089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] PATH_INFO: `/v3/projects/b0738a12e3794e2b9a45beb56d6eeb09` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.530566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0738a12e3794e2b9a45beb56d6eeb09)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.534263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.586800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24009a4d4cab47d4933a16755b42b4bb', 'request_id': 'req-789a3db9-8a69-43a6-8e66-5e9ab390ece6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.597166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24009a4d4cab47d4933a16755b42b4bb', 'request_id': 'req-789a3db9-8a69-43a6-8e66-5e9ab390ece6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.598728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0738a12e3794e2b9a45beb56d6eeb09', 'request_id': 'req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.606684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24009a4d4cab47d4933a16755b42b4bb', 'request_id': 'req-789a3db9-8a69-43a6-8e66-5e9ab390ece6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.608512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0738a12e3794e2b9a45beb56d6eeb09', 'request_id': 'req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.616477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0738a12e3794e2b9a45beb56d6eeb09', 'request_id': 'req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.622451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] The token cache is being invalidate because project 24009a4d4cab47d4933a16755b42b4bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:06.622893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.624257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-789a3db9-8a69-43a6-8e66-5e9ab390ece6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.631271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1034/2052] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:06 2022] DELETE /identity/v3/projects/24009a4d4cab47d4933a16755b42b4bb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:06.632380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] The token cache is being invalidate because project b0738a12e3794e2b9a45beb56d6eeb09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:06.632670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.632945 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdace4-c562-4013-ad39-147bc641d552 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.633243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdace4-c562-4013-ad39-147bc641d552 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.633376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8d3e522d-3ebc-4d6b-b46a-3384dcaf0e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:06.633545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdace4-c562-4013-ad39-147bc641d552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.635124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1019/2053] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:06 2022] DELETE /identity/v3/projects/b0738a12e3794e2b9a45beb56d6eeb09 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:06.640238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380f5f5-4167-42a7-89ed-ebec5ed7cc0f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.640591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380f5f5-4167-42a7-89ed-ebec5ed7cc0f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.640889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380f5f5-4167-42a7-89ed-ebec5ed7cc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.653912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-3efdace4-c562-4013-ad39-147bc641d552 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:06.655178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-3efdace4-c562-4013-ad39-147bc641d552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:06.659776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c380f5f5-4167-42a7-89ed-ebec5ed7cc0f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:06.660983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c380f5f5-4167-42a7-89ed-ebec5ed7cc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:06.714181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1035/2054] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:06.724460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6fe10-7f6e-4269-94e8-2481bec23936 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.724824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6fe10-7f6e-4269-94e8-2481bec23936 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.725112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6fe10-7f6e-4269-94e8-2481bec23936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.732474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1020/2055] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:06.740252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8a99-50ef-4983-83ef-b786bf9d5f52 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.740542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8a99-50ef-4983-83ef-b786bf9d5f52 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.740851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8a99-50ef-4983-83ef-b786bf9d5f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.753672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-41c6fe10-7f6e-4269-94e8-2481bec23936 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:06.755296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-41c6fe10-7f6e-4269-94e8-2481bec23936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:06.761776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6bca8a99-50ef-4983-83ef-b786bf9d5f52 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:06.767010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6bca8a99-50ef-4983-83ef-b786bf9d5f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:06.769269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1036/2056] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:06.775049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6fe10-7f6e-4269-94e8-2481bec23936 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.776813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1021/2057] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:06.781110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca8a99-50ef-4983-83ef-b786bf9d5f52 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.784659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6fe10-7f6e-4269-94e8-2481bec23936 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.790865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.791020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca8a99-50ef-4983-83ef-b786bf9d5f52 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.791765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.792439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.792947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.794338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:06.794651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.796256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.796922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.797052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.797333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.797618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.798422 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:06.798730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.800419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.810807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1037/2058] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:54:06.817472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1022/2059] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:06.817642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.818648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.827615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b9b21-8189-4793-8623-6220c165342f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.828689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0d7c4-efa8-4ded-8a6d-7859f78724f8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.833911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.834092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.834576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.834858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.835131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.835473 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.835825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.836104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.836519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.837438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.837815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.839818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.866101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1023/2060] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:06.870446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.875832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1038/2061] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:06.876923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.943990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970eea2-cf5e-450e-bf17-f276e827ef14 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.947638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12ae9-8907-4048-9a95-8c82fbbfcd88 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.949627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.950514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.950769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.951044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.951749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.952784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.953523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.953970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.954102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.954102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.954738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:06.956844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:06.977299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1024/2062] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:54:06.981076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.983209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1039/2063] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:06 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:06.984126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:06.992373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9258d-f56b-4ab9-9654-adf40b2ff66d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.993767 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb0034-0d60-43d0-84a4-9c91d49b3692 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:06.997755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.998423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.998530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:06.998704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.998977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.999078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:06.999347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:06.999619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:06.999727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.000361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.001378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.002243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.011969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1025/2064] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:07.013506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1040/2065] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:07.016410 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:07.018928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:07.027403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266271e-7314-4925-b902-16ca5de652b5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:07.030995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f41a-4f9e-4200-8120-20988e21ae44 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:07.032106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:07.032811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.033185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.033361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.034085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.035680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.036954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:07.037649 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.037923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.038190 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.038923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.040527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.051000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1026/2066] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:07.051917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:07.053754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1041/2067] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:07.058110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:07.063090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6f20-1c8b-47a7-add9-61e6a101bb37 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:07.067833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93daf0d-ff3f-4588-81db-29d5a4133f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:07.068836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93daf0d-ff3f-4588-81db-29d5a4133f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.069186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6db217-72b2-4492-be15-aa4f7e4d3f19 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:07.069319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93daf0d-ff3f-4588-81db-29d5a4133f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.069573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93daf0d-ff3f-4588-81db-29d5a4133f02 admin admin] PATH_INFO: `/v3/projects/697957bb3e1d4d57a52e327602ebcedf/users/fa93c3828a4a4daea1a31971d4cadda7/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.074539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd48690-41ac-4868-be63-cb335c9dce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:07.075543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd48690-41ac-4868-be63-cb335c9dce01 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.075808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd48690-41ac-4868-be63-cb335c9dce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.076966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd48690-41ac-4868-be63-cb335c9dce01 admin admin] PATH_INFO: `/v3/projects/38df2edd247e4400a7a9ca3704faab1a/users/da79e0772ca1446295643e4688ea219b/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.092236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93daf0d-ff3f-4588-81db-29d5a4133f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=697957bb3e1d4d57a52e327602ebcedf, user_id=fa93c3828a4a4daea1a31971d4cadda7, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.095545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93daf0d-ff3f-4588-81db-29d5a4133f02 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.099428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd48690-41ac-4868-be63-cb335c9dce01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38df2edd247e4400a7a9ca3704faab1a, user_id=da79e0772ca1446295643e4688ea219b, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.102298 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd48690-41ac-4868-be63-cb335c9dce01 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.116647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1027/2068] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:07 2022] PUT /identity/v3/projects/697957bb3e1d4d57a52e327602ebcedf/users/fa93c3828a4a4daea1a31971d4cadda7/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:07.118287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1042/2069] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:07 2022] PUT /identity/v3/projects/38df2edd247e4400a7a9ca3704faab1a/users/da79e0772ca1446295643e4688ea219b/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:07.128486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.128884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.129230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.129342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.129540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.129962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.158624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] MFA Rules not processed for user `fa93c3828a4a4daea1a31971d4cadda7`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:07.161849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:07.164059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] MFA Rules not processed for user `da79e0772ca1446295643e4688ea219b`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:07.165578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:07.236573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1028/2070] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:07.243508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1043/2071] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:07.248975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:07.250211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:07.255294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:07.256708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:07.323702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e120b8a-e355-4c9b-bcb5-02dfe0343868 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:07.324047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9b52c-1bf5-4cfa-a465-bb3c0d2c78e3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:07.329133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:07.329917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.330199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.330467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.330622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4af5f-21c5-4959-bd26-d1eea33ce5e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:07.331832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4af5f-21c5-4959-bd26-d1eea33ce5e5 service glance] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:07.332250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4af5f-21c5-4959-bd26-d1eea33ce5e5 service glance] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:07.332871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4af5f-21c5-4959-bd26-d1eea33ce5e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:07.341952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.343493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.346295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4af5f-21c5-4959-bd26-d1eea33ce5e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:07.347925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4af5f-21c5-4959-bd26-d1eea33ce5e5 service glance] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:07.353295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1029/2072] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:07.359406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1044/2073] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:12.230288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:12.232437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:12.438607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c252b8-9c3e-4aac-a0f5-3ab170e20daf service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:12.440295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:12.440518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:12.441158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:12.441438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] PATH_INFO: `/v3/users/2b4b2d6563804db8a1043f1fd2951001` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:12.442982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4b2d6563804db8a1043f1fd2951001)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:12.445669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:12.501544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4b2d6563804db8a1043f1fd2951001'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:12.508687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4b2d6563804db8a1043f1fd2951001'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:12.508687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4b2d6563804db8a1043f1fd2951001'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:12.516984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4b2d6563804db8a1043f1fd2951001'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:12.555925 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4b2d6563804db8a1043f1fd2951001'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:12.578957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1030/2074] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:12 2022] DELETE /identity/v3/users/2b4b2d6563804db8a1043f1fd2951001 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:12.584114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:12.584772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:12.584772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:12.618765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:12.620157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:12.707960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1045/2075] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:12.722703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:12.724423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:12.854061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853f7f5-fb75-4200-9522-878320848c98 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:12.860348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71feaffd-affb-4fa4-a17a-4696b34a3473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:12.861184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feaffd-affb-4fa4-a17a-4696b34a3473 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:12.861493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feaffd-affb-4fa4-a17a-4696b34a3473 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:12.861765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71feaffd-affb-4fa4-a17a-4696b34a3473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:12.874496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feaffd-affb-4fa4-a17a-4696b34a3473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:12.875915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71feaffd-affb-4fa4-a17a-4696b34a3473 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:12.888501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1031/2076] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:13.474914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:13.479412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:13.521418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536db463-b43f-4364-a27a-7e569d45dd45 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:13.532902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:13.534104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:13.534457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:13.534881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] PATH_INFO: `/v3/projects/60b5344da0a24c9bb99773903c762f14` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:13.536730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b5344da0a24c9bb99773903c762f14)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:13.538841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:13.639884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b5344da0a24c9bb99773903c762f14', 'request_id': 'req-a406f33b-ac03-406e-9510-6030dc36d4b8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:13.653202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b5344da0a24c9bb99773903c762f14', 'request_id': 'req-a406f33b-ac03-406e-9510-6030dc36d4b8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:13.661494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b5344da0a24c9bb99773903c762f14', 'request_id': 'req-a406f33b-ac03-406e-9510-6030dc36d4b8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:13.682444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] The token cache is being invalidate because project 60b5344da0a24c9bb99773903c762f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:13.682813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:13.683424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a406f33b-ac03-406e-9510-6030dc36d4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:13.691570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1046/2077] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:13 2022] DELETE /identity/v3/projects/60b5344da0a24c9bb99773903c762f14 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:13.695181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2c05-7074-4caf-80fb-9b250619d1fa None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:13.695951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2c05-7074-4caf-80fb-9b250619d1fa None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:13.695951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2c05-7074-4caf-80fb-9b250619d1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:13.743548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7d2a2c05-7074-4caf-80fb-9b250619d1fa None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:13.747430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7d2a2c05-7074-4caf-80fb-9b250619d1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:13.872777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1032/2078] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:13.882487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456e2e2-abf2-4688-9d07-7e693145f7cd None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:13.883025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456e2e2-abf2-4688-9d07-7e693145f7cd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:13.883364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456e2e2-abf2-4688-9d07-7e693145f7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:13.913738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8456e2e2-abf2-4688-9d07-7e693145f7cd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:13.915330 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8456e2e2-abf2-4688-9d07-7e693145f7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:13.940414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1047/2079] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:13.944075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a2c05-7074-4caf-80fb-9b250619d1fa None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:13.963685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a2c05-7074-4caf-80fb-9b250619d1fa None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:13.985196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:13.986922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:13.987413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:13.988003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:13.989170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:13.989691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:13.993162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:14.033707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1033/2080] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:14.038574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456e2e2-abf2-4688-9d07-7e693145f7cd None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:14.062777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456e2e2-abf2-4688-9d07-7e693145f7cd None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:14.069801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:14.070809 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:14.071267 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:14.072782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:14.074331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:14.076409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:14.127822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1048/2081] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:14 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:14.132481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:14.145399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc9fa8-1940-4bc3-abe8-956f92c9109e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:14.221458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:14.222439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:14.223000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:14.223375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:14.224126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:14.226540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:14.257896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1034/2082] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:14 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:14.262590 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:14.278856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a72d7-a232-48cd-ba1a-bfe838499a1f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:14.286509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:14.287547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:14.287837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:14.288118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:14.289491 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:14.292410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:14.308840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1049/2083] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:14.313761 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:14.335627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3dc60-68be-4e3a-b836-d7b0a1057d56 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:14.347902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2226a-fb14-47d1-bfbd-94682f2f2335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:14.349723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2226a-fb14-47d1-bfbd-94682f2f2335 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:14.350481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2226a-fb14-47d1-bfbd-94682f2f2335 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:14.351216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2226a-fb14-47d1-bfbd-94682f2f2335 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:14.353499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2226a-fb14-47d1-bfbd-94682f2f2335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:14.358306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2226a-fb14-47d1-bfbd-94682f2f2335 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:14.404118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1035/2084] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:14.410559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:14.431442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c707579-0f45-41a4-82dc-3d6ea4bd0abd admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:14.441237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:14.442731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:14.443285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:14.443756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] PATH_INFO: `/v3/projects/7b7f4521072e438baac42d80040f7a0c/users/0c26df4446d04d21be3581f21dfbb3ae/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:14.476138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b7f4521072e438baac42d80040f7a0c, user_id=0c26df4446d04d21be3581f21dfbb3ae, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:14.481335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:14.506651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1050/2085] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:14 2022] PUT /identity/v3/projects/7b7f4521072e438baac42d80040f7a0c/users/0c26df4446d04d21be3581f21dfbb3ae/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:14.529233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5325516-42b9-4224-b86e-01cdb6db7bf0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:14.529686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5325516-42b9-4224-b86e-01cdb6db7bf0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:14.530136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5325516-42b9-4224-b86e-01cdb6db7bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:14.593627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d5325516-42b9-4224-b86e-01cdb6db7bf0 None None] MFA Rules not processed for user `0c26df4446d04d21be3581f21dfbb3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:14.595571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d5325516-42b9-4224-b86e-01cdb6db7bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:14.713865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1036/2086] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:14.733908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:14.737732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:14.837571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31473ad-e19e-43e8-8b41-144ba1d53da7 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:14.845899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa3108-bc1d-4259-9650-45a0af9cfa95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:14.847016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa3108-bc1d-4259-9650-45a0af9cfa95 service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:14.847525 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa3108-bc1d-4259-9650-45a0af9cfa95 service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:14.848032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa3108-bc1d-4259-9650-45a0af9cfa95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:14.862767 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa3108-bc1d-4259-9650-45a0af9cfa95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:14.865893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa3108-bc1d-4259-9650-45a0af9cfa95 service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:14.880341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1051/2087] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:15.110232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5325516-42b9-4224-b86e-01cdb6db7bf0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:15.178041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5325516-42b9-4224-b86e-01cdb6db7bf0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:15.183981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:15.184913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:15.185277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:15.185557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] PATH_INFO: `/v3/users/0c26df4446d04d21be3581f21dfbb3ae` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:15.186891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c26df4446d04d21be3581f21dfbb3ae)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:15.188940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:15.247910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c26df4446d04d21be3581f21dfbb3ae'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:15.248604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c26df4446d04d21be3581f21dfbb3ae'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:15.249313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c26df4446d04d21be3581f21dfbb3ae'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:15.256838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c26df4446d04d21be3581f21dfbb3ae'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:15.278541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c26df4446d04d21be3581f21dfbb3ae'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:15.303571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1037/2088] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:15 2022] DELETE /identity/v3/users/0c26df4446d04d21be3581f21dfbb3ae => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:15.308327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:15.308689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:15.309209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:15.330417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:15.331702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:15.410637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1052/2089] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:15.420911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:15.422501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:15.490065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde28485-44f8-4dc1-86a7-10ade5f5527e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:15.495440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d481f8-40f6-4add-b1fc-4d27267ed856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:15.496531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d481f8-40f6-4add-b1fc-4d27267ed856 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:15.497129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d481f8-40f6-4add-b1fc-4d27267ed856 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:15.497609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d481f8-40f6-4add-b1fc-4d27267ed856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:15.514594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d481f8-40f6-4add-b1fc-4d27267ed856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:15.516148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d481f8-40f6-4add-b1fc-4d27267ed856 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:15.528861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1038/2090] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:16.007664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:16.010490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:16.060626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86968665-337e-4fd5-ade8-34d6f1ffa0dd None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:16.076322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:16.077443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.077724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.078029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] PATH_INFO: `/v3/projects/7b7f4521072e438baac42d80040f7a0c` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.079495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b7f4521072e438baac42d80040f7a0c)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:16.081962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:16.150322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b7f4521072e438baac42d80040f7a0c', 'request_id': 'req-e414d820-c85f-4603-99ff-ead13799944c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:16.161571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b7f4521072e438baac42d80040f7a0c', 'request_id': 'req-e414d820-c85f-4603-99ff-ead13799944c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:16.171592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b7f4521072e438baac42d80040f7a0c', 'request_id': 'req-e414d820-c85f-4603-99ff-ead13799944c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:16.190880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] The token cache is being invalidate because project 7b7f4521072e438baac42d80040f7a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:16.191529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:16.192694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e414d820-c85f-4603-99ff-ead13799944c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:16.195667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1053/2091] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:16 2022] DELETE /identity/v3/projects/7b7f4521072e438baac42d80040f7a0c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:16.202037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55372e-4b77-46a2-82b7-9c0820cb0848 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.202599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55372e-4b77-46a2-82b7-9c0820cb0848 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.202734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55372e-4b77-46a2-82b7-9c0820cb0848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.222857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-1a55372e-4b77-46a2-82b7-9c0820cb0848 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:16.224091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-1a55372e-4b77-46a2-82b7-9c0820cb0848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:16.301345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1039/2092] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:16.311439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e49120-08b9-4526-9309-5e7b852e76f3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.311873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e49120-08b9-4526-9309-5e7b852e76f3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.312326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e49120-08b9-4526-9309-5e7b852e76f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.341597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a9e49120-08b9-4526-9309-5e7b852e76f3 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:16.342902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a9e49120-08b9-4526-9309-5e7b852e76f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:16.352147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1054/2093] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:16.359895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55372e-4b77-46a2-82b7-9c0820cb0848 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:16.371551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55372e-4b77-46a2-82b7-9c0820cb0848 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:16.377161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:16.378128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.378479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.378803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.379649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:16.380060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:16.382881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:16.401304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1040/2094] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:16.407647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e49120-08b9-4526-9309-5e7b852e76f3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:16.421720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e49120-08b9-4526-9309-5e7b852e76f3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:16.430102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:16.431251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.431736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.432285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.434178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:16.438273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:16.476104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1055/2095] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:16 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:16.480447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:16.517257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8bbc-98d7-427c-a9b1-12e3d767e523 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:16.630608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:16.631311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.631600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.631893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.632824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:16.636050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:16.668834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1041/2096] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:16 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:16.673285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:16.688215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4bc9a-0629-4ce6-92f9-cab10fd3829d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:16.695049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:16.695825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.696303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.696486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.697361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:16.699063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:16.713933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1056/2097] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:16.718642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:16.731806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c48f1-0dff-40a8-84aa-81426ffaa3cc admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:16.737050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe288a33-a251-480f-8348-b81acbffc5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:16.737748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe288a33-a251-480f-8348-b81acbffc5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.738030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe288a33-a251-480f-8348-b81acbffc5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.738307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe288a33-a251-480f-8348-b81acbffc5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.739058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe288a33-a251-480f-8348-b81acbffc5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:16.740784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe288a33-a251-480f-8348-b81acbffc5a9 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:16.756245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1042/2098] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:16.763974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:16.780931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf8656-d005-4856-a03a-1c0326a38aee admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:16.787179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:16.788088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.788370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.788645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] PATH_INFO: `/v3/projects/3992a0a7278140cb8f56d40304b67e3c/users/f16364278ea84897a1118c52894a23b5/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.817075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3992a0a7278140cb8f56d40304b67e3c, user_id=f16364278ea84897a1118c52894a23b5, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:16.819780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:16.837160 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1057/2099] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:16 2022] PUT /identity/v3/projects/3992a0a7278140cb8f56d40304b67e3c/users/f16364278ea84897a1118c52894a23b5/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:16.851789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5121fe-baab-4560-b72b-02106453c18f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:16.852038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5121fe-baab-4560-b72b-02106453c18f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:16.852395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5121fe-baab-4560-b72b-02106453c18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:16.883026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5d5121fe-baab-4560-b72b-02106453c18f None None] MFA Rules not processed for user `f16364278ea84897a1118c52894a23b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:16.884279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5d5121fe-baab-4560-b72b-02106453c18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:16.961879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1043/2100] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:16.966684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:17.055197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a0a51-4abd-472d-af95-3a0df5bc358f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:17.063094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:17.063771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.064080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.064371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.065073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:17.068002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:17.100917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1058/2101] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:16 2022] POST /identity/v3/projects => generated 361 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:17.105834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5121fe-baab-4560-b72b-02106453c18f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:17.122824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5121fe-baab-4560-b72b-02106453c18f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:17.197015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:17.197967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.198327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.198693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.199508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:17.201591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:17.239337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1044/2102] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:17 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:17.246813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:17.265678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014e188-824f-44c1-88ca-5c8ff029dd0a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:17.273433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:17.274506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.274948 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.275402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.276634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:17.279758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:17.294845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1059/2103] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:17.299838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:17.314954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd9904-044a-42a6-9979-28d34d949184 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:17.336665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:17.338302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.338759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.339216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] PATH_INFO: `/v3/projects/729c0fc787914b8592b4873ee0782772/users/749fc02b0a9a4591a6e9f7a9eada47b6/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.364745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=729c0fc787914b8592b4873ee0782772, user_id=749fc02b0a9a4591a6e9f7a9eada47b6, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:17.370335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:17.388788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1045/2104] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:17 2022] PUT /identity/v3/projects/729c0fc787914b8592b4873ee0782772/users/749fc02b0a9a4591a6e9f7a9eada47b6/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:17.402080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:17.415559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea9304-df0e-4ad1-9162-d10f6d18878b admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:17.487855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4717e-4cb3-44ad-a0d2-7c47f5fc0f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:17.488630 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4717e-4cb3-44ad-a0d2-7c47f5fc0f22 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.489034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4717e-4cb3-44ad-a0d2-7c47f5fc0f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.489209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4717e-4cb3-44ad-a0d2-7c47f5fc0f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.490003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4717e-4cb3-44ad-a0d2-7c47f5fc0f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:17.491829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4717e-4cb3-44ad-a0d2-7c47f5fc0f22 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:17.503206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1060/2105] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:17.507976 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:17.519343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f5339-a957-4a07-8f47-a4d6fc2ecf7b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:17.524631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:17.525882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.526164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.526439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] PATH_INFO: `/v3/projects/729c0fc787914b8592b4873ee0782772/users/749fc02b0a9a4591a6e9f7a9eada47b6/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.529304 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=729c0fc787914b8592b4873ee0782772, user_id=749fc02b0a9a4591a6e9f7a9eada47b6, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:17.532509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:17.547016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1046/2106] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:17 2022] PUT /identity/v3/projects/729c0fc787914b8592b4873ee0782772/users/749fc02b0a9a4591a6e9f7a9eada47b6/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:17.558436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48c264-fb79-4af5-843a-66afa01d3cf5 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.558732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48c264-fb79-4af5-843a-66afa01d3cf5 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.559047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48c264-fb79-4af5-843a-66afa01d3cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.588170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4d48c264-fb79-4af5-843a-66afa01d3cf5 None None] MFA Rules not processed for user `749fc02b0a9a4591a6e9f7a9eada47b6`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:17.589388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4d48c264-fb79-4af5-843a-66afa01d3cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:17.684632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1061/2107] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:17 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:17.703339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:17.705899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:17.784995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c386de09-7f1c-4a5e-93ae-b0e77943f868 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:17.788883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:17.789699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:17.789911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:17.790171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:17.809347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:17.810952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:17.822647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1047/2108] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:17 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:18.321664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48c264-fb79-4af5-843a-66afa01d3cf5 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:18.423857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48c264-fb79-4af5-843a-66afa01d3cf5 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:18.434377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:18.435319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:18.435591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:18.435925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] PATH_INFO: `/v3/users/f16364278ea84897a1118c52894a23b5` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:18.437463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16364278ea84897a1118c52894a23b5)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:18.439467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:18.463577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:18.465895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:18.498519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c00a17-f2ca-4c98-9d0e-1cd7dd4b14f4 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:18.505612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16364278ea84897a1118c52894a23b5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.507331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:18.509180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:18.510056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:18.510829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] PATH_INFO: `/v3/users/fa93c3828a4a4daea1a31971d4cadda7` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:18.514200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa93c3828a4a4daea1a31971d4cadda7)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:18.528508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:18.528782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16364278ea84897a1118c52894a23b5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.528782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16364278ea84897a1118c52894a23b5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.528782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16364278ea84897a1118c52894a23b5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.536270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fae3b7d8-fc0f-4e45-be84-3d3c81e38dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16364278ea84897a1118c52894a23b5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.556074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1062/2109] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:18 2022] DELETE /identity/v3/users/f16364278ea84897a1118c52894a23b5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:18.566709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:18.568126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:18.569163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:18.611797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:18.614601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:18.630106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa93c3828a4a4daea1a31971d4cadda7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.632588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa93c3828a4a4daea1a31971d4cadda7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.633719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa93c3828a4a4daea1a31971d4cadda7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.648334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa93c3828a4a4daea1a31971d4cadda7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.681583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-73e67297-022c-4735-a677-9f867edbf0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa93c3828a4a4daea1a31971d4cadda7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:18.727462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1048/2110] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:18 2022] DELETE /identity/v3/users/fa93c3828a4a4daea1a31971d4cadda7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:18.737329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:18.738015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:18.738609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:18.743619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1063/2111] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:18.757463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:18.765315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:18.778056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:18.780255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:18.895870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1049/2112] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:18.906461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af265d0-5ec5-4fb3-a122-5d79a1bfe259 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:18.923151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:18.927447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:19.025623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a4420-0903-48ba-b744-d845a5884196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:19.026654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a4420-0903-48ba-b744-d845a5884196 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.027119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a4420-0903-48ba-b744-d845a5884196 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.028210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a4420-0903-48ba-b744-d845a5884196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.044599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a4420-0903-48ba-b744-d845a5884196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:19.047361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a4420-0903-48ba-b744-d845a5884196 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:19.065162 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1064/2113] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:19.067664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.068203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.068684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.073295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0780-7287-48a0-a869-1869bef8568c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:19.081989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:19.083912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.084425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.084962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.111577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:19.114758 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] MFA Rules not processed for user `7533ae6ea6774f7eb853455d086f939d`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:19.115336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:19.116972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:19.135403 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1050/2114] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:19.221871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1065/2115] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:19.510666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:19.514207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:19.536678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cbeeb-14f9-46a7-845f-1a20fee83788 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:19.544982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:19.546681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.547238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.547907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] PATH_INFO: `/v3/projects/697957bb3e1d4d57a52e327602ebcedf` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.550384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] RBAC: Authorizing `identity:delete_project(project_id=697957bb3e1d4d57a52e327602ebcedf)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:19.556888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:19.627073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '697957bb3e1d4d57a52e327602ebcedf', 'request_id': 'req-39755feb-518d-42f8-9dd4-c41b769acbea'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.629218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:19.631954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:19.635190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '697957bb3e1d4d57a52e327602ebcedf', 'request_id': 'req-39755feb-518d-42f8-9dd4-c41b769acbea'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.649754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '697957bb3e1d4d57a52e327602ebcedf', 'request_id': 'req-39755feb-518d-42f8-9dd4-c41b769acbea'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.667243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] The token cache is being invalidate because project 697957bb3e1d4d57a52e327602ebcedf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:19.667725 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.668517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-39755feb-518d-42f8-9dd4-c41b769acbea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.671589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1051/2116] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:19 2022] DELETE /identity/v3/projects/697957bb3e1d4d57a52e327602ebcedf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:19.678672 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1a0c4-3c03-4336-82ab-a6b6789d7c75 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.679096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac627401-0386-40c7-8660-e3c6a438cfbe None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:19.680080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1a0c4-3c03-4336-82ab-a6b6789d7c75 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.680372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1a0c4-3c03-4336-82ab-a6b6789d7c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.710722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3ab1a0c4-3c03-4336-82ab-a6b6789d7c75 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:19.711926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3ab1a0c4-3c03-4336-82ab-a6b6789d7c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:19.749300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:19.750151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.750441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.750818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] PATH_INFO: `/v3/projects/3992a0a7278140cb8f56d40304b67e3c` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.752378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3992a0a7278140cb8f56d40304b67e3c)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:19.754583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:19.788230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1052/2117] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:19.796446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388877a0-4afe-4073-805f-1e433d60ba39 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.796733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388877a0-4afe-4073-805f-1e433d60ba39 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.797176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388877a0-4afe-4073-805f-1e433d60ba39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.819160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-388877a0-4afe-4073-805f-1e433d60ba39 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:19.821095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-388877a0-4afe-4073-805f-1e433d60ba39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:19.826626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3992a0a7278140cb8f56d40304b67e3c', 'request_id': 'req-f63ea412-31e1-4393-ab8c-0d220db7efc2'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.842338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3992a0a7278140cb8f56d40304b67e3c', 'request_id': 'req-f63ea412-31e1-4393-ab8c-0d220db7efc2'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.851501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3992a0a7278140cb8f56d40304b67e3c', 'request_id': 'req-f63ea412-31e1-4393-ab8c-0d220db7efc2'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.873709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] The token cache is being invalidate because project 3992a0a7278140cb8f56d40304b67e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:19.874003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.877433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:19.880367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1066/2118] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:19 2022] DELETE /identity/v3/projects/3992a0a7278140cb8f56d40304b67e3c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:19.886757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:19.893228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:19.911865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1053/2119] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:19.916769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388877a0-4afe-4073-805f-1e433d60ba39 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:19.928578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388877a0-4afe-4073-805f-1e433d60ba39 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:19.935964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:19.936364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.936767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.936977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.938087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:19.938466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:19.943825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:19.969271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1054/2120] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:54:19.970699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:19.983418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c350b-b985-427b-9366-7ed7d11253cf admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:19.991499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:19.992513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:19.992990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:19.993425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:19.994166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:19.996125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:20.001636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea412-31e1-4393-ab8c-0d220db7efc2 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:20.006646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:20.007721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:20.008011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:20.008294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] PATH_INFO: `/v3/users/749fc02b0a9a4591a6e9f7a9eada47b6` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:20.009686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=749fc02b0a9a4591a6e9f7a9eada47b6)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:20.011726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:20.052158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1055/2121] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:19 2022] POST /identity/v3/projects => generated 389 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:54:20.054398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:20.099904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '749fc02b0a9a4591a6e9f7a9eada47b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:20.126272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '749fc02b0a9a4591a6e9f7a9eada47b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:20.127612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '749fc02b0a9a4591a6e9f7a9eada47b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:20.128664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '749fc02b0a9a4591a6e9f7a9eada47b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:20.142366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '749fc02b0a9a4591a6e9f7a9eada47b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:20.156059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27f457-abcf-4c69-a5f1-8718c6e27a8c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:20.162199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:20.163083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:20.163473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:20.163857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:20.164916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:20.167071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:20.167934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1067/2122] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:19 2022] DELETE /identity/v3/users/749fc02b0a9a4591a6e9f7a9eada47b6 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:20.203914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1056/2123] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:20 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:20.208788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:20.212425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:20.431503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df16583-a0fe-413d-9227-0ad011d85c01 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:20.440299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:20.441757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:20.442263 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:20.443496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:20.447634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:20.448117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:20.473700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1068/2124] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:20.477468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:20.495631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c65c8b-ab32-4386-9041-fa9ff74180b1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:20.503876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19af58f-7e56-483e-97d3-ad031c664981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:20.505781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af58f-7e56-483e-97d3-ad031c664981 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:20.506181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af58f-7e56-483e-97d3-ad031c664981 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:20.506555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af58f-7e56-483e-97d3-ad031c664981 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:20.514319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19af58f-7e56-483e-97d3-ad031c664981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:20.516500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19af58f-7e56-483e-97d3-ad031c664981 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:20.530562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1057/2125] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:20.534747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:20.554474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef71ba-9c91-4266-976e-3e8808fb1b91 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:20.564493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:20.566902 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:20.567509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:20.567977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] PATH_INFO: `/v3/projects/395a8279ba6b4ffc829bcce8c2022584/users/f1a8de1c829d48118dc15251fd84cbe0/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:20.599508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395a8279ba6b4ffc829bcce8c2022584, user_id=f1a8de1c829d48118dc15251fd84cbe0, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:20.605183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:20.633112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1069/2126] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:20 2022] PUT /identity/v3/projects/395a8279ba6b4ffc829bcce8c2022584/users/f1a8de1c829d48118dc15251fd84cbe0/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:20.641795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:20.642081 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:20.642402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:20.678703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] MFA Rules not processed for user `f1a8de1c829d48118dc15251fd84cbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:20.680309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:20.806406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1058/2127] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:20.829823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:20.831713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:20.869459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:20.871149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:20.950285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b52e24-22ed-445c-8ffd-ab4ef1310316 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:20.971268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8300df-ca4e-442a-ba2f-d83a07266f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:20.971954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8300df-ca4e-442a-ba2f-d83a07266f7f service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:20.972227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8300df-ca4e-442a-ba2f-d83a07266f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:20.972678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8300df-ca4e-442a-ba2f-d83a07266f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:20.985728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8300df-ca4e-442a-ba2f-d83a07266f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:20.987759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8300df-ca4e-442a-ba2f-d83a07266f7f service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:20.999684 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf4cd7-8380-43ff-9033-1956af290cf7 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:21.001218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1070/2128] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:21.012175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:21.017895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.018181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.018501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] PATH_INFO: `/v3/projects/729c0fc787914b8592b4873ee0782772` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.020301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=729c0fc787914b8592b4873ee0782772)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:21.022438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:21.103565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '729c0fc787914b8592b4873ee0782772', 'request_id': 'req-9bbaa474-f709-42c2-8551-4c5c82f7bb72'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:21.111099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '729c0fc787914b8592b4873ee0782772', 'request_id': 'req-9bbaa474-f709-42c2-8551-4c5c82f7bb72'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:21.119252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '729c0fc787914b8592b4873ee0782772', 'request_id': 'req-9bbaa474-f709-42c2-8551-4c5c82f7bb72'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:21.143849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] The token cache is being invalidate because project 729c0fc787914b8592b4873ee0782772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:21.144553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:21.145366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-9bbaa474-f709-42c2-8551-4c5c82f7bb72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:21.147577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1059/2129] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:20 2022] DELETE /identity/v3/projects/729c0fc787914b8592b4873ee0782772 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:21.152763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83e085-9a8e-45ca-be8b-b74c836b9486 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.153147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83e085-9a8e-45ca-be8b-b74c836b9486 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.153425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83e085-9a8e-45ca-be8b-b74c836b9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.176021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ef83e085-9a8e-45ca-be8b-b74c836b9486 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:21.177571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ef83e085-9a8e-45ca-be8b-b74c836b9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:21.276055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1071/2130] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:21.278998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47319b6-bde1-4012-9ff2-e6118644f211 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.279510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47319b6-bde1-4012-9ff2-e6118644f211 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.279926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47319b6-bde1-4012-9ff2-e6118644f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.310708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d47319b6-bde1-4012-9ff2-e6118644f211 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:21.311986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d47319b6-bde1-4012-9ff2-e6118644f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:21.330659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1060/2131] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:21.341322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83e085-9a8e-45ca-be8b-b74c836b9486 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:21.358010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83e085-9a8e-45ca-be8b-b74c836b9486 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:21.369350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:21.371281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.372009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.372765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.375343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:21.376118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:21.381315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:21.421764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1072/2132] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:21.445022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47319b6-bde1-4012-9ff2-e6118644f211 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:21.458949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47319b6-bde1-4012-9ff2-e6118644f211 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:21.473018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:21.474166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.474717 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.475212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.476651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:21.481458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:21.538346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1061/2133] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:21 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:21.549396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:21.567898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23038f77-57bb-42c1-a08b-c0f1994b26e5 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:21.672836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:21.674539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.675013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.675790 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.678075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:21.682904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:21.721401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1073/2134] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:21 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:21.731082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:21.743800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bd2be-0225-43b9-ba0f-67beb765c256 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:21.749654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:21.750342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.750751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.751037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.751824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:21.754020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:21.770810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1062/2135] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:21.776089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:21.789570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7985039-57fb-4571-9c46-56b9368f35e7 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:21.799577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c08b3f-188a-4b36-a461-172d41572643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:21.800847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c08b3f-188a-4b36-a461-172d41572643 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.801245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c08b3f-188a-4b36-a461-172d41572643 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.801617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c08b3f-188a-4b36-a461-172d41572643 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.802552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c08b3f-188a-4b36-a461-172d41572643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:21.804618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c08b3f-188a-4b36-a461-172d41572643 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:21.819103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1074/2136] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:21.823578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:21.844349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645e9c-6b31-46f9-beb0-8c212f930fe3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:21.860279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:21.861982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.862498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.862852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] PATH_INFO: `/v3/projects/545c1443423e48e298ff4c58b7f0da43/users/3bbdaf52c47849d8a249381fd9ebeb81/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:21.896537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=545c1443423e48e298ff4c58b7f0da43, user_id=3bbdaf52c47849d8a249381fd9ebeb81, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:21.900183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:21.930381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1063/2137] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:21 2022] PUT /identity/v3/projects/545c1443423e48e298ff4c58b7f0da43/users/3bbdaf52c47849d8a249381fd9ebeb81/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:21.949270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecb3b8-954a-4028-93c0-1b296ecede38 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:21.949923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecb3b8-954a-4028-93c0-1b296ecede38 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:21.951183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecb3b8-954a-4028-93c0-1b296ecede38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.004422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-86ecb3b8-954a-4028-93c0-1b296ecede38 None None] MFA Rules not processed for user `3bbdaf52c47849d8a249381fd9ebeb81`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:22.007621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-86ecb3b8-954a-4028-93c0-1b296ecede38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:22.119629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1075/2138] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:22.125771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:22.210518 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf86910-7b4d-4497-bd48-e77bf04388a8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:22.220771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:22.222150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:22.222749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:22.223291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.224777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:22.230593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:22.293171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1064/2139] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:22 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:22.294453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecb3b8-954a-4028-93c0-1b296ecede38 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:22.310636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecb3b8-954a-4028-93c0-1b296ecede38 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:22.402547 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:22.403337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:22.403691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:22.404037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.404860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:22.407117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:22.441598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1076/2140] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:22 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:22.443748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:22.475862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c61061-fa96-49b3-93db-adea93333a89 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:22.484241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:22.485275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:22.485562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:22.485849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.486625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:22.488315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:22.506775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1065/2141] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:22.513458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:22.533238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9f655-7fce-4d6d-9b3d-3a81a7725202 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:22.543011 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:22.544618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:22.545178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:22.545665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] PATH_INFO: `/v3/projects/787fa38007504c0983cbfbbea0db1ad7/users/0d54026e81f04a45ab1ca7ca094cdd66/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.586515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787fa38007504c0983cbfbbea0db1ad7, user_id=0d54026e81f04a45ab1ca7ca094cdd66, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:22.591117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:22.623089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1077/2142] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:22 2022] PUT /identity/v3/projects/787fa38007504c0983cbfbbea0db1ad7/users/0d54026e81f04a45ab1ca7ca094cdd66/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:22.627830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:22.643375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273f1dc-f92a-4d61-9612-2f9981a4370f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:22.741486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9fbc2-3728-47f0-9257-093dffdbb440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:22.742581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9fbc2-3728-47f0-9257-093dffdbb440 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:22.743167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9fbc2-3728-47f0-9257-093dffdbb440 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:22.743755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9fbc2-3728-47f0-9257-093dffdbb440 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.745753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9fbc2-3728-47f0-9257-093dffdbb440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:22.749161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9fbc2-3728-47f0-9257-093dffdbb440 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:22.768999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1066/2143] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:22.774564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:22.789871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f751-3d87-4cd4-a493-1c7d18c670f2 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:22.799394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:22.801302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:22.801914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:22.802253 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] PATH_INFO: `/v3/projects/787fa38007504c0983cbfbbea0db1ad7/users/0d54026e81f04a45ab1ca7ca094cdd66/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.807353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=787fa38007504c0983cbfbbea0db1ad7, user_id=0d54026e81f04a45ab1ca7ca094cdd66, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:22.812065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:22.833458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1078/2144] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:22 2022] PUT /identity/v3/projects/787fa38007504c0983cbfbbea0db1ad7/users/0d54026e81f04a45ab1ca7ca094cdd66/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:22.854005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e1356-0ecf-464e-9304-b84a83d62b85 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:22.854660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e1356-0ecf-464e-9304-b84a83d62b85 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:22.855663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e1356-0ecf-464e-9304-b84a83d62b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:22.916158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-586e1356-0ecf-464e-9304-b84a83d62b85 None None] MFA Rules not processed for user `0d54026e81f04a45ab1ca7ca094cdd66`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:22.918345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-586e1356-0ecf-464e-9304-b84a83d62b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:23.033713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1067/2145] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:23.049480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:23.051476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:23.168864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc163f85-3def-47e1-944c-cee668931b3d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:23.175749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:23.176879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:23.177711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:23.177939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:23.190833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:23.193584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:23.206783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1079/2146] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:23.305559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e1356-0ecf-464e-9304-b84a83d62b85 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:23.321743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e1356-0ecf-464e-9304-b84a83d62b85 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:23.325824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace22511-caa3-4481-b957-1c25bf0d3c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:23.326665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22511-caa3-4481-b957-1c25bf0d3c07 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:23.327040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22511-caa3-4481-b957-1c25bf0d3c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:23.327406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22511-caa3-4481-b957-1c25bf0d3c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:23.399320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace22511-caa3-4481-b957-1c25bf0d3c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:23.401341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace22511-caa3-4481-b957-1c25bf0d3c07 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:23.412109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1068/2147] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:27.343289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:27.406720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea0ab5-eb18-45fd-a3af-3dbd94d28b83 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:27.412507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:27.413583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:27.413868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:27.414230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] PATH_INFO: `/v3/users/3bbdaf52c47849d8a249381fd9ebeb81` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:27.415585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbdaf52c47849d8a249381fd9ebeb81)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:27.417524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:27.465643 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbdaf52c47849d8a249381fd9ebeb81'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:27.476596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbdaf52c47849d8a249381fd9ebeb81'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:27.477336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbdaf52c47849d8a249381fd9ebeb81'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:27.477943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbdaf52c47849d8a249381fd9ebeb81'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:27.486742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbdaf52c47849d8a249381fd9ebeb81'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:27.498682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1080/2148] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:27 2022] DELETE /identity/v3/users/3bbdaf52c47849d8a249381fd9ebeb81 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:27.503198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:27.503666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:27.503799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:27.520987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:27.522173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:27.581728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1069/2149] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:27.591339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:27.592774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:27.655703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf7231-f1dc-49db-825a-554581a89aec admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:27.662212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:27.663782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:27.664332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:27.664887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:27.681796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:27.684786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:27.706572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1081/2150] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:28.085610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:28.087321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:28.100611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b3d9a-75ee-43b7-abce-d4aa5406f991 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:28.106479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:28.107484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:28.107773 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:28.108108 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] PATH_INFO: `/v3/projects/545c1443423e48e298ff4c58b7f0da43` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:28.109799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=545c1443423e48e298ff4c58b7f0da43)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:28.112167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:28.165415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545c1443423e48e298ff4c58b7f0da43', 'request_id': 'req-70688f77-54f3-414b-89e8-27edb0f11bbe'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.172693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545c1443423e48e298ff4c58b7f0da43', 'request_id': 'req-70688f77-54f3-414b-89e8-27edb0f11bbe'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.179574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545c1443423e48e298ff4c58b7f0da43', 'request_id': 'req-70688f77-54f3-414b-89e8-27edb0f11bbe'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.193023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] The token cache is being invalidate because project 545c1443423e48e298ff4c58b7f0da43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:28.193339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.193955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.195825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1070/2151] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:28 2022] DELETE /identity/v3/projects/545c1443423e48e298ff4c58b7f0da43 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:28.199720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:28.209689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d71b49-0896-4bc8-899d-9e132ddff5a7 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:28.275058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:28.275936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:28.276212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:28.276483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] PATH_INFO: `/v3/users/0d54026e81f04a45ab1ca7ca094cdd66` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:28.277753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d54026e81f04a45ab1ca7ca094cdd66)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:28.279658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:28.351250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d54026e81f04a45ab1ca7ca094cdd66'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.366247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d54026e81f04a45ab1ca7ca094cdd66'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.367094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d54026e81f04a45ab1ca7ca094cdd66'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.367774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d54026e81f04a45ab1ca7ca094cdd66'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.379168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23cb0f11-515a-475b-97f5-4b8c60abcaf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d54026e81f04a45ab1ca7ca094cdd66'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.396283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1082/2152] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:28 2022] DELETE /identity/v3/users/0d54026e81f04a45ab1ca7ca094cdd66 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:28.735662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:28.737358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:28.752386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70688f77-54f3-414b-89e8-27edb0f11bbe admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:28.812869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:28.813738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:28.814005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:28.814269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] PATH_INFO: `/v3/projects/787fa38007504c0983cbfbbea0db1ad7` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:28.815484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787fa38007504c0983cbfbbea0db1ad7)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:28.817432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:28.869746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787fa38007504c0983cbfbbea0db1ad7', 'request_id': 'req-c798184f-3367-4fa3-bf07-078ac6287f06'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.875918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787fa38007504c0983cbfbbea0db1ad7', 'request_id': 'req-c798184f-3367-4fa3-bf07-078ac6287f06'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.881283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787fa38007504c0983cbfbbea0db1ad7', 'request_id': 'req-c798184f-3367-4fa3-bf07-078ac6287f06'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.895224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] The token cache is being invalidate because project 787fa38007504c0983cbfbbea0db1ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:28.895506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.896124 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c798184f-3367-4fa3-bf07-078ac6287f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:28.898033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1071/2153] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:28 2022] DELETE /identity/v3/projects/787fa38007504c0983cbfbbea0db1ad7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:28.902806 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bd5d2-929d-491c-a687-4dfa44ea6abd None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:28.903113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bd5d2-929d-491c-a687-4dfa44ea6abd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:28.903414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bd5d2-929d-491c-a687-4dfa44ea6abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:28.934257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4d5bd5d2-929d-491c-a687-4dfa44ea6abd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:28.935484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4d5bd5d2-929d-491c-a687-4dfa44ea6abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:28.998771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1083/2154] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:29.008146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba591ffd-db79-407c-ac6d-bf2796731d7f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.008405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba591ffd-db79-407c-ac6d-bf2796731d7f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.008732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba591ffd-db79-407c-ac6d-bf2796731d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.032461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ba591ffd-db79-407c-ac6d-bf2796731d7f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:29.033768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ba591ffd-db79-407c-ac6d-bf2796731d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:29.043167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1072/2155] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:29.047998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bd5d2-929d-491c-a687-4dfa44ea6abd None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.059124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bd5d2-929d-491c-a687-4dfa44ea6abd None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.065784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.066589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.066934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.067366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.068365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:29.068623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.071300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.085475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1084/2156] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:29.090781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba591ffd-db79-407c-ac6d-bf2796731d7f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.100959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba591ffd-db79-407c-ac6d-bf2796731d7f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.107318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.108014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.108294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.108566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.109341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.111285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.145966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1073/2157] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:29 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:29.147079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.157701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00af6d-f278-4e9e-8862-84420d98e29f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.222913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.224039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.224536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.225054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.226370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.228669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.256334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1085/2158] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:29 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:54:29.260693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.274059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26c632-a11b-4e4b-bb20-9fdb027dc193 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.279193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.280867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.280867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.280867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.282037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.283688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.295929 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1074/2159] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:29.300393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.312182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45a103-372e-4719-a607-ceb66955e257 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.317746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b7ded-08fa-4ee3-af56-de3a39aeb968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.318564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b7ded-08fa-4ee3-af56-de3a39aeb968 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.318854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b7ded-08fa-4ee3-af56-de3a39aeb968 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.319144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b7ded-08fa-4ee3-af56-de3a39aeb968 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.319966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b7ded-08fa-4ee3-af56-de3a39aeb968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.321745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b7ded-08fa-4ee3-af56-de3a39aeb968 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.334841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1086/2160] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:29.339747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.351017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1cc7f-3a8d-4d74-9e66-2ef3f3ad7ae3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.356868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.357951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.358247 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.358526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] PATH_INFO: `/v3/projects/67204cc994b14fa49accdbe355853a19/users/b764201266e74692b2ccfcfd174cdbda/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.374690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=67204cc994b14fa49accdbe355853a19, user_id=b764201266e74692b2ccfcfd174cdbda, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.377425 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.393465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1075/2161] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:29 2022] PUT /identity/v3/projects/67204cc994b14fa49accdbe355853a19/users/b764201266e74692b2ccfcfd174cdbda/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:29.404392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f9976-77c2-4e5b-8379-e4528949654c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.405128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f9976-77c2-4e5b-8379-e4528949654c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.405128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f9976-77c2-4e5b-8379-e4528949654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.434043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-018f9976-77c2-4e5b-8379-e4528949654c None None] MFA Rules not processed for user `b764201266e74692b2ccfcfd174cdbda`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:29.435259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-018f9976-77c2-4e5b-8379-e4528949654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:29.518167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1087/2162] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:29.523079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.586860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760304e-7ba7-40de-9189-627b13b127cf admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.593057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.593995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.594283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.594733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.595597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.597907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.644991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1076/2163] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:29 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:54:29.648206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f9976-77c2-4e5b-8379-e4528949654c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.667519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f9976-77c2-4e5b-8379-e4528949654c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.754914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.755617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.755900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.756171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.756927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.758867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.781405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1088/2164] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:29 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:54:29.785462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.797426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee0c36-7ae1-470d-8228-d2540f4cc609 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.805832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.806528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.806810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.807085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.807831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.811361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.822931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1077/2165] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:29.827046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.837370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5afc2-c744-4a07-8306-4378013dfb90 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.842212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb4515-841a-438f-bce8-136d6b443216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.842872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4515-841a-438f-bce8-136d6b443216 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.843143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4515-841a-438f-bce8-136d6b443216 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.843413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4515-841a-438f-bce8-136d6b443216 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.844149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb4515-841a-438f-bce8-136d6b443216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.845768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb4515-841a-438f-bce8-136d6b443216 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.856769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1089/2166] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:29.861089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:29.872736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dae3f-abf9-4884-a9c3-6e7224018fec admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:29.877945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:29.878830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.879106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.879372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] PATH_INFO: `/v3/projects/fbb696bf11604184878bf63f6a15e364/users/e51661c3b76e4ddd8c4ed6e1a7fa3313/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.895285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb696bf11604184878bf63f6a15e364, user_id=e51661c3b76e4ddd8c4ed6e1a7fa3313, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:29.898105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:29.913787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1078/2167] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:29 2022] PUT /identity/v3/projects/fbb696bf11604184878bf63f6a15e364/users/e51661c3b76e4ddd8c4ed6e1a7fa3313/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:29.924906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651a05-7e6e-43f5-bf30-8ad8321f0167 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:29.925130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651a05-7e6e-43f5-bf30-8ad8321f0167 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:29.925408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651a05-7e6e-43f5-bf30-8ad8321f0167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:29.953218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b0651a05-7e6e-43f5-bf30-8ad8321f0167 None None] MFA Rules not processed for user `e51661c3b76e4ddd8c4ed6e1a7fa3313`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:29.954434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b0651a05-7e6e-43f5-bf30-8ad8321f0167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:30.027444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1090/2168] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:30.032082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.101805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e7439-c59e-4ca3-a693-4e640e4776c9 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.107617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.108297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.108569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.108861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.109557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.111453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.139555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1079/2169] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:30 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:30.143802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651a05-7e6e-43f5-bf30-8ad8321f0167 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.154580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651a05-7e6e-43f5-bf30-8ad8321f0167 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.216327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.216930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.217214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.217480 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.218188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.220151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.241665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1091/2170] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:30 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:30.245788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.256511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c817-1ac0-4fd2-b666-e43bca9bed41 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.262029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.263038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.263315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.263652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.264387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.266037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.278430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1080/2171] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:30.283839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.305309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d8127-d2d4-44ea-bda9-87b723b7c434 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.324957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.326557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.327056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.327492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] PATH_INFO: `/v3/projects/c5becd0e2a134ea181090f08718ca252/users/714195e4eaab4e89972210f211211344/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.350913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5becd0e2a134ea181090f08718ca252, user_id=714195e4eaab4e89972210f211211344, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.353662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.369376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1092/2172] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:30 2022] PUT /identity/v3/projects/c5becd0e2a134ea181090f08718ca252/users/714195e4eaab4e89972210f211211344/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:30.373447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.384088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa13b6-3b2c-4a45-853b-8ee74560e5fe admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.448289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8168c-03c1-4673-bfcd-981ba5724731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.448939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8168c-03c1-4673-bfcd-981ba5724731 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.449314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8168c-03c1-4673-bfcd-981ba5724731 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.449775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8168c-03c1-4673-bfcd-981ba5724731 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.450329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8168c-03c1-4673-bfcd-981ba5724731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.452004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8168c-03c1-4673-bfcd-981ba5724731 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.463120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1081/2173] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:30.468495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.481875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a82b92-239d-4f0a-8a3d-44792c4d265c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.487935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.489099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.489482 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.489873 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] PATH_INFO: `/v3/projects/c5becd0e2a134ea181090f08718ca252/users/714195e4eaab4e89972210f211211344/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.492184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5becd0e2a134ea181090f08718ca252, user_id=714195e4eaab4e89972210f211211344, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.495198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.507040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1093/2174] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:30 2022] PUT /identity/v3/projects/c5becd0e2a134ea181090f08718ca252/users/714195e4eaab4e89972210f211211344/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:30.524011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.524349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.524628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.554314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] MFA Rules not processed for user `714195e4eaab4e89972210f211211344`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:30.555739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:30.627310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1082/2175] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:30.639161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.641207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:30.704786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75502d9-1865-4dc4-aded-c99c4529a2b8 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.708602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.709322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.709577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.709853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.720249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.721620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.732579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1094/2176] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:30.792376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:30.794787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:30.816069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9119d84-170c-43c9-9149-02c3c2e24bd6 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:30.822966 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4db0b-ba7d-4bc7-8d10-a87604f4c8ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:30.823952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4db0b-ba7d-4bc7-8d10-a87604f4c8ea service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:30.824450 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4db0b-ba7d-4bc7-8d10-a87604f4c8ea service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:30.825052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4db0b-ba7d-4bc7-8d10-a87604f4c8ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:30.895070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4db0b-ba7d-4bc7-8d10-a87604f4c8ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:30.896622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4db0b-ba7d-4bc7-8d10-a87604f4c8ea service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:30.906619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1083/2177] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:35.120403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:35.122104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:35.191531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b3b64-d0f1-444c-b61a-a63355f3e113 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:35.197246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:35.198154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:35.198429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:35.198713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] PATH_INFO: `/v3/users/da79e0772ca1446295643e4688ea219b` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:35.200065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da79e0772ca1446295643e4688ea219b)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:35.202071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:35.253305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da79e0772ca1446295643e4688ea219b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:35.266287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da79e0772ca1446295643e4688ea219b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:35.267013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da79e0772ca1446295643e4688ea219b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:35.267652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da79e0772ca1446295643e4688ea219b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:35.277333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da79e0772ca1446295643e4688ea219b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:35.291679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1095/2178] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:35 2022] DELETE /identity/v3/users/da79e0772ca1446295643e4688ea219b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:35.296768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:35.297325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:35.297325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:35.316473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:35.317727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:35.385135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1084/2179] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:35.395891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:35.397934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:35.462329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799278fa-b8c3-4b3b-924d-824a0b660291 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:35.466268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd00a7-5604-4943-a2a9-a57a4bf64bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:35.466972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd00a7-5604-4943-a2a9-a57a4bf64bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:35.467249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd00a7-5604-4943-a2a9-a57a4bf64bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:35.467517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd00a7-5604-4943-a2a9-a57a4bf64bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:35.478107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd00a7-5604-4943-a2a9-a57a4bf64bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:35.479566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd00a7-5604-4943-a2a9-a57a4bf64bc6 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:35.494382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1096/2180] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:35.882048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:35.884077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:35.906016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf003a8-6188-4aba-b07c-b88c1591ef78 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:35.912149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:35.913068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:35.913352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:35.913628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] PATH_INFO: `/v3/projects/38df2edd247e4400a7a9ca3704faab1a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:35.915610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38df2edd247e4400a7a9ca3704faab1a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:35.918216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.001230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38df2edd247e4400a7a9ca3704faab1a', 'request_id': 'req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.015769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38df2edd247e4400a7a9ca3704faab1a', 'request_id': 'req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.025185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38df2edd247e4400a7a9ca3704faab1a', 'request_id': 'req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.044927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] The token cache is being invalidate because project 38df2edd247e4400a7a9ca3704faab1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:36.045378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.046156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.048187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1085/2181] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:35 2022] DELETE /identity/v3/projects/38df2edd247e4400a7a9ca3704faab1a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:36.057498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc52b5-d38a-4acc-ae23-5a96e40cc666 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.057787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc52b5-d38a-4acc-ae23-5a96e40cc666 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.058269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc52b5-d38a-4acc-ae23-5a96e40cc666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.062138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.064857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:36.081240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a8cc52b5-d38a-4acc-ae23-5a96e40cc666 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:36.082830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a8cc52b5-d38a-4acc-ae23-5a96e40cc666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:36.148512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a175a3-7d28-4c7b-8fe2-712dd976e5a3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.156314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.158084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.158559 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.159000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] PATH_INFO: `/v3/users/b764201266e74692b2ccfcfd174cdbda` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.160930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b764201266e74692b2ccfcfd174cdbda)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.162931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.164596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1097/2182] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:36.173578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a9014-f810-458a-a0d8-96b8bb17cc20 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.174283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a9014-f810-458a-a0d8-96b8bb17cc20 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.174563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a9014-f810-458a-a0d8-96b8bb17cc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.194836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e82a9014-f810-458a-a0d8-96b8bb17cc20 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:36.196086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e82a9014-f810-458a-a0d8-96b8bb17cc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:36.217564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1098/2183] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:36.219057 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a9014-f810-458a-a0d8-96b8bb17cc20 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.229111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b764201266e74692b2ccfcfd174cdbda'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.229601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a9014-f810-458a-a0d8-96b8bb17cc20 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.233460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b764201266e74692b2ccfcfd174cdbda'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.234404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b764201266e74692b2ccfcfd174cdbda'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.234752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.235510 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.235868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.236200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.237050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:36.237489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.239170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.244284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b764201266e74692b2ccfcfd174cdbda'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.252662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1099/2184] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:36.257096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.260753 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:36.265613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ac3fb21a-8b0c-4e40-8f51-6a4efef6969c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b764201266e74692b2ccfcfd174cdbda'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:36.285791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758d40b-e795-4be1-b4a8-43cc92ec9c82 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.286092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1086/2185] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:36 2022] DELETE /identity/v3/users/b764201266e74692b2ccfcfd174cdbda => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:36.290681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe49b-3d6a-457d-9194-cdf97341cc3c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.290973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe49b-3d6a-457d-9194-cdf97341cc3c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.291252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe49b-3d6a-457d-9194-cdf97341cc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.313548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-8a6fe49b-3d6a-457d-9194-cdf97341cc3c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:36.314859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8a6fe49b-3d6a-457d-9194-cdf97341cc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:36.355109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.356105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.356105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.356301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.357146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.359174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.401765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1100/2186] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:36 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:54:36.409094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.411462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:36.412122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1087/2187] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:36.421973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fe49b-3d6a-457d-9194-cdf97341cc3c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.452246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fe49b-3d6a-457d-9194-cdf97341cc3c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.511430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c5b41-96c0-4ce6-8316-426a8b5dc53b admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.523252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.523922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.524198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.524469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.606184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.606916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.607197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.607473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.608397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.610545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.639068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.640854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.641384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1088/2188] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:36 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:36.647033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.651865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1101/2189] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:36.666977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17896d0e-14c8-4f58-81bc-b56900eacd02 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.683151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.685878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.686415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.686743 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.688388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.690956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.705363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1089/2190] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:36.709299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.741427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2155-9933-4a7f-b985-9b8a4f450040 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.765045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37603-7342-43cf-8406-533ce777a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.766192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37603-7342-43cf-8406-533ce777a57b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.770368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37603-7342-43cf-8406-533ce777a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.770980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37603-7342-43cf-8406-533ce777a57b admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.778660 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37603-7342-43cf-8406-533ce777a57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.789175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37603-7342-43cf-8406-533ce777a57b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.838210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:36.839446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1102/2191] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:36.857090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cec22-ce31-459c-8095-a54698ed0a6d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:36.862939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:36.863859 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.864140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.864461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] PATH_INFO: `/v3/projects/b3a6e4e824d5437299c2c0fca3e83d66/users/1f3244c979ba4c72a21b6c7426fcc8d8/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:36.893692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6e4e824d5437299c2c0fca3e83d66, user_id=1f3244c979ba4c72a21b6c7426fcc8d8, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:36.898633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:36.944835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1090/2192] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:36 2022] PUT /identity/v3/projects/b3a6e4e824d5437299c2c0fca3e83d66/users/1f3244c979ba4c72a21b6c7426fcc8d8/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:36.947366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:36.947901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:36.948333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:37.053663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] MFA Rules not processed for user `1f3244c979ba4c72a21b6c7426fcc8d8`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:37.055428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:37.192857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1103/2193] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:37.205157 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:37.207188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:37.285208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07645c63-b7f5-4117-82b8-04e9846b7294 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:37.289428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:37.290175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:37.290513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:37.290854 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:37.300757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:37.302372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:37.312758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1091/2194] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:37.348868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:37.350342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:37.427294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24d06c-1917-4e80-83d8-00b058602487 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:37.433092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:37.433565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:37.433814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:37.434096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] PATH_INFO: `/v3/projects/67204cc994b14fa49accdbe355853a19` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:37.435754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67204cc994b14fa49accdbe355853a19)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:37.437967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:37.501148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67204cc994b14fa49accdbe355853a19', 'request_id': 'req-9f908492-1a12-45a3-8f79-3562cf1f5b04'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.514275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67204cc994b14fa49accdbe355853a19', 'request_id': 'req-9f908492-1a12-45a3-8f79-3562cf1f5b04'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.527483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67204cc994b14fa49accdbe355853a19', 'request_id': 'req-9f908492-1a12-45a3-8f79-3562cf1f5b04'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.544192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] The token cache is being invalidate because project 67204cc994b14fa49accdbe355853a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:37.545058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.546641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.550586 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1104/2195] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:37 2022] DELETE /identity/v3/projects/67204cc994b14fa49accdbe355853a19 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:37.557835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:37.578296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cef97-2c31-4f72-aed0-79d2b9e05462 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:37.660155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:37.661140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:37.661610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:37.661915 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] PATH_INFO: `/v3/users/e51661c3b76e4ddd8c4ed6e1a7fa3313` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:37.663237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e51661c3b76e4ddd8c4ed6e1a7fa3313)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:37.665284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:37.743413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e51661c3b76e4ddd8c4ed6e1a7fa3313'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.744147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e51661c3b76e4ddd8c4ed6e1a7fa3313'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.744885 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e51661c3b76e4ddd8c4ed6e1a7fa3313'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.753349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e51661c3b76e4ddd8c4ed6e1a7fa3313'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.768863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e51661c3b76e4ddd8c4ed6e1a7fa3313'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:37.784240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1092/2196] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:37 2022] DELETE /identity/v3/users/e51661c3b76e4ddd8c4ed6e1a7fa3313 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:38.137604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:38.139952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:38.187666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f908492-1a12-45a3-8f79-3562cf1f5b04 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:38.285296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:38.286942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:38.287366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:38.287777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] PATH_INFO: `/v3/projects/fbb696bf11604184878bf63f6a15e364` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:38.289620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb696bf11604184878bf63f6a15e364)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:38.292160 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:38.364967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb696bf11604184878bf63f6a15e364', 'request_id': 'req-f636f397-3537-4390-9d43-820405d265ee'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.373808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb696bf11604184878bf63f6a15e364', 'request_id': 'req-f636f397-3537-4390-9d43-820405d265ee'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.380246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb696bf11604184878bf63f6a15e364', 'request_id': 'req-f636f397-3537-4390-9d43-820405d265ee'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.394963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] The token cache is being invalidate because project fbb696bf11604184878bf63f6a15e364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:38.395423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.396340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.398185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1105/2197] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:38 2022] DELETE /identity/v3/projects/fbb696bf11604184878bf63f6a15e364 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:38.402834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:38.405958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:38.428194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22031863-59d2-4cec-897a-634fba0c4927 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:38.505777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:38.507501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:38.507787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:38.508064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] PATH_INFO: `/v3/users/714195e4eaab4e89972210f211211344` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:38.509621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714195e4eaab4e89972210f211211344)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:38.511575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:38.568411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714195e4eaab4e89972210f211211344'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.569114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714195e4eaab4e89972210f211211344'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.569720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714195e4eaab4e89972210f211211344'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.586481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714195e4eaab4e89972210f211211344'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.600115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4df9781c-dcc0-4b3f-b038-7be759c5d422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714195e4eaab4e89972210f211211344'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:38.613992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1093/2198] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:38 2022] DELETE /identity/v3/users/714195e4eaab4e89972210f211211344 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:39.382231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:39.385545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:39.400212 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636f397-3537-4390-9d43-820405d265ee admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:39.464929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:39.465845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.466196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.466496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] PATH_INFO: `/v3/projects/c5becd0e2a134ea181090f08718ca252` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.467837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5becd0e2a134ea181090f08718ca252)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:39.471459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:39.520446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5becd0e2a134ea181090f08718ca252', 'request_id': 'req-fcad7bac-eec1-44e1-98d7-77156eacdea5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:39.535472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5becd0e2a134ea181090f08718ca252', 'request_id': 'req-fcad7bac-eec1-44e1-98d7-77156eacdea5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:39.542957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5becd0e2a134ea181090f08718ca252', 'request_id': 'req-fcad7bac-eec1-44e1-98d7-77156eacdea5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:39.557402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] The token cache is being invalidate because project c5becd0e2a134ea181090f08718ca252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:39.557402 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:39.558190 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fcad7bac-eec1-44e1-98d7-77156eacdea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:39.559958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1106/2199] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:39 2022] DELETE /identity/v3/projects/c5becd0e2a134ea181090f08718ca252 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:39.566972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288eda7-75be-44d1-8a2a-d18e66af6cef None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.567336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288eda7-75be-44d1-8a2a-d18e66af6cef None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.567671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288eda7-75be-44d1-8a2a-d18e66af6cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.588030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7288eda7-75be-44d1-8a2a-d18e66af6cef None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:39.589387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7288eda7-75be-44d1-8a2a-d18e66af6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:39.662831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1094/2200] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:39.671269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6007c6-b019-4f2f-8eb3-1ab9a7e04197 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.671558 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6007c6-b019-4f2f-8eb3-1ab9a7e04197 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.671833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6007c6-b019-4f2f-8eb3-1ab9a7e04197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.690351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6e6007c6-b019-4f2f-8eb3-1ab9a7e04197 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:39.691565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6e6007c6-b019-4f2f-8eb3-1ab9a7e04197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:39.700101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1107/2201] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:39.704639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288eda7-75be-44d1-8a2a-d18e66af6cef None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:39.714746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288eda7-75be-44d1-8a2a-d18e66af6cef None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:39.719844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:39.720525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.720829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.721274 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.722055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:39.722368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:39.724030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:39.738327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1095/2202] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:39.743251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6007c6-b019-4f2f-8eb3-1ab9a7e04197 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:39.753997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6007c6-b019-4f2f-8eb3-1ab9a7e04197 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:39.758759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:39.759478 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.759759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.760027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.760724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:39.762977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:39.790332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1108/2203] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:39 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:39.794903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:39.805638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e209776-1757-4d16-89a0-e41d17d932d4 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:39.872615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:39.873227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.873507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.873782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.874497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:39.876426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:39.898206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1096/2204] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:39 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:39.904209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:39.916540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa96c3-870c-4a8a-9e80-2861d54fae7f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:39.922506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:39.923191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.923471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.923746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.924488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:39.926185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:39.938199 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1109/2205] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:39.942301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:39.954036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74d64c-e7d0-499e-8d17-f704e0b0863f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:39.961572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882a2a7-f1b3-475b-ac3a-928ab519ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:39.962438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882a2a7-f1b3-475b-ac3a-928ab519ce7a admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:39.962758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882a2a7-f1b3-475b-ac3a-928ab519ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:39.963075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882a2a7-f1b3-475b-ac3a-928ab519ce7a admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:39.964146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882a2a7-f1b3-475b-ac3a-928ab519ce7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:39.966642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882a2a7-f1b3-475b-ac3a-928ab519ce7a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:39.980110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1097/2206] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:39.986354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:39.996671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26de9f-6cf6-4e6d-959b-5120dffe11d6 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.001927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.002838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.003111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.003378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] PATH_INFO: `/v3/projects/ef4a60c7e89b4c31b273afb63248a7ff/users/ee679bcf98b149db99ad6b191954b4b0/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.021237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4a60c7e89b4c31b273afb63248a7ff, user_id=ee679bcf98b149db99ad6b191954b4b0, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.024750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.040931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1110/2207] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:39 2022] PUT /identity/v3/projects/ef4a60c7e89b4c31b273afb63248a7ff/users/ee679bcf98b149db99ad6b191954b4b0/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:40.054138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fd86-5b15-420a-b1ab-aca1b5c06df3 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.054387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fd86-5b15-420a-b1ab-aca1b5c06df3 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.054695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82fd86-5b15-420a-b1ab-aca1b5c06df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.085221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7a82fd86-5b15-420a-b1ab-aca1b5c06df3 None None] MFA Rules not processed for user `ee679bcf98b149db99ad6b191954b4b0`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:40.086478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7a82fd86-5b15-420a-b1ab-aca1b5c06df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:40.166916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1098/2208] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:40.171992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.239658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46ef6f-2692-4325-af87-ba9ee4f7b059 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.245647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.246509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.246793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.247075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.247969 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.250100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.289998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1111/2209] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:40 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:40.298687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82fd86-5b15-420a-b1ab-aca1b5c06df3 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.310102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82fd86-5b15-420a-b1ab-aca1b5c06df3 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.397171 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.398206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.398676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.399116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.400490 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.404040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.427090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1099/2210] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:40 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:40.431103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.442717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16a7cb-77b1-4b91-9257-0519fb545504 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.447904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.448670 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.449041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.449318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.450269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.452319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.471382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1112/2211] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:40.472097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.482596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a46d7-3a53-4f94-827d-a89cbab2cc05 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.487598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb1779-e5f6-4f9b-8d52-cf596ea439fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.488268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb1779-e5f6-4f9b-8d52-cf596ea439fd admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.488553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb1779-e5f6-4f9b-8d52-cf596ea439fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.489181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb1779-e5f6-4f9b-8d52-cf596ea439fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.489739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb1779-e5f6-4f9b-8d52-cf596ea439fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.491342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb1779-e5f6-4f9b-8d52-cf596ea439fd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.505620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1100/2212] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:40.507054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.518406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca2c09-f737-4ad6-88a7-70966c6899d8 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.532339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.532339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.532339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.532339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] PATH_INFO: `/v3/projects/4f7f6e5e499745bc99307a032d37f159/users/aea8ab77da0146fcadde15d08e901f2e/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.546369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f7f6e5e499745bc99307a032d37f159, user_id=aea8ab77da0146fcadde15d08e901f2e, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.549306 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.565580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1113/2213] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:40 2022] PUT /identity/v3/projects/4f7f6e5e499745bc99307a032d37f159/users/aea8ab77da0146fcadde15d08e901f2e/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:40.576654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc390d-0396-4308-8a01-7c25dcc33153 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.576990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc390d-0396-4308-8a01-7c25dcc33153 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.577273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc390d-0396-4308-8a01-7c25dcc33153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.607560 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5efc390d-0396-4308-8a01-7c25dcc33153 None None] MFA Rules not processed for user `aea8ab77da0146fcadde15d08e901f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:40.608766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5efc390d-0396-4308-8a01-7c25dcc33153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:40.679424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1101/2214] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:40.684362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.752234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280b5a3-30c7-4014-98f0-a4fb42f7fb3d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.757322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.758034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.758314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.758594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.759319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.761285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.800337 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1114/2215] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:40 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:40.804485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc390d-0396-4308-8a01-7c25dcc33153 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.816562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc390d-0396-4308-8a01-7c25dcc33153 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.886746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b66c62-af07-42d9-8869-656667858add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.888172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b66c62-af07-42d9-8869-656667858add admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.888586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b66c62-af07-42d9-8869-656667858add admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.889060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b66c62-af07-42d9-8869-656667858add admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.889885 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b66c62-af07-42d9-8869-656667858add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.892193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b66c62-af07-42d9-8869-656667858add admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.919587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1102/2216] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:40 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:54:40.924696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.938560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4eb1-f9d2-4751-91c6-30939ffd7879 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.946159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.947242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.947717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.948188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:40.949518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:40.952538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:40.967138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1115/2217] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:40.971752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b66c62-af07-42d9-8869-656667858add admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:40.982453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b66c62-af07-42d9-8869-656667858add admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:40.987428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:40.988306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:40.988595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:40.988897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] PATH_INFO: `/v3/projects/63ac0280ead94c46a3ee42fe1ec01cca/users/f5c68af9d89543539fc1e32f321c1c8b/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:41.005562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ac0280ead94c46a3ee42fe1ec01cca, user_id=f5c68af9d89543539fc1e32f321c1c8b, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:41.008628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:41.024563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1103/2218] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:40 2022] PUT /identity/v3/projects/63ac0280ead94c46a3ee42fe1ec01cca/users/f5c68af9d89543539fc1e32f321c1c8b/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:41.029086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:41.040194 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996b92a-ffeb-43a1-8043-ce1ba79cda29 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:41.104113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae8ce1-afec-4908-9f04-92d51a60501c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:41.104834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8ce1-afec-4908-9f04-92d51a60501c admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:41.105100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8ce1-afec-4908-9f04-92d51a60501c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:41.105373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae8ce1-afec-4908-9f04-92d51a60501c admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:41.106109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae8ce1-afec-4908-9f04-92d51a60501c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:41.107713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae8ce1-afec-4908-9f04-92d51a60501c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:41.117967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1116/2219] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:41.123117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:41.133610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fa761-5e44-4c15-a68c-f33cd36b5ec5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:41.139314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:41.140202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:41.140482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:41.140756 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] PATH_INFO: `/v3/projects/63ac0280ead94c46a3ee42fe1ec01cca/users/f5c68af9d89543539fc1e32f321c1c8b/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:41.142816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ac0280ead94c46a3ee42fe1ec01cca, user_id=f5c68af9d89543539fc1e32f321c1c8b, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:41.145547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:41.156338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1104/2220] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:41 2022] PUT /identity/v3/projects/63ac0280ead94c46a3ee42fe1ec01cca/users/f5c68af9d89543539fc1e32f321c1c8b/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:41.166956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562d04-277b-49d5-9567-66001963af5c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:41.167243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562d04-277b-49d5-9567-66001963af5c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:41.167516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562d04-277b-49d5-9567-66001963af5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:41.199173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6b562d04-277b-49d5-9567-66001963af5c None None] MFA Rules not processed for user `f5c68af9d89543539fc1e32f321c1c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:41.200380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6b562d04-277b-49d5-9567-66001963af5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:41.286334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1117/2221] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:42.381531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:42.383259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:42.521580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84b526-9bf8-4973-8bf5-9c0b903c688e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:42.527956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f8417-573f-40e4-8752-75e7961e4d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:42.528940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f8417-573f-40e4-8752-75e7961e4d13 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:42.529449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f8417-573f-40e4-8752-75e7961e4d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:42.529696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f8417-573f-40e4-8752-75e7961e4d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:42.598135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f8417-573f-40e4-8752-75e7961e4d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:42.599821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f8417-573f-40e4-8752-75e7961e4d13 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:42.612036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1105/2222] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:43.721442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562d04-277b-49d5-9567-66001963af5c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:43.722862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6b562d04-277b-49d5-9567-66001963af5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:43.801329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562d04-277b-49d5-9567-66001963af5c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:43.806038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:43.806916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:43.807416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:43.807690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] PATH_INFO: `/v3/users/f1a8de1c829d48118dc15251fd84cbe0` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:43.808971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a8de1c829d48118dc15251fd84cbe0)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:43.811055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:43.862651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a8de1c829d48118dc15251fd84cbe0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:43.875373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a8de1c829d48118dc15251fd84cbe0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:43.876124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a8de1c829d48118dc15251fd84cbe0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:43.876726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a8de1c829d48118dc15251fd84cbe0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:43.884990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a8de1c829d48118dc15251fd84cbe0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:43.903176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1118/2223] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:43 2022] DELETE /identity/v3/users/f1a8de1c829d48118dc15251fd84cbe0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:43.909567 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:43.910309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:43.910662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:43.934201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:43.936537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:44.002405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1106/2224] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:44.013183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:44.015034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:44.085290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e56dd-7700-45ff-acda-7d8c8d5aa1d4 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:44.096290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:44.096290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:44.096290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:44.096290 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:44.105051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:44.106591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:44.118496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1119/2225] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:44.284260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:44.285909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:44.360426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e95c60-df6c-441b-8696-727b26fd34ab None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:44.365070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:44.365747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:44.366031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:44.366305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:44.368674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:44.444011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:44.445493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:44.456277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1107/2226] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:44.556363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:44.558213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:44.578929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7616bce-8c2c-41c4-99ef-3a038f99f3ed service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:44.585987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:44.586989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:44.587363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:44.587729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] PATH_INFO: `/v3/projects/395a8279ba6b4ffc829bcce8c2022584` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:44.589171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=395a8279ba6b4ffc829bcce8c2022584)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:44.591860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:44.659934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395a8279ba6b4ffc829bcce8c2022584', 'request_id': 'req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:44.670834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395a8279ba6b4ffc829bcce8c2022584', 'request_id': 'req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:44.678543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395a8279ba6b4ffc829bcce8c2022584', 'request_id': 'req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:44.690791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] The token cache is being invalidate because project 395a8279ba6b4ffc829bcce8c2022584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:44.691177 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:44.691757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:44.693786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1120/2227] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:44 2022] DELETE /identity/v3/projects/395a8279ba6b4ffc829bcce8c2022584 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:45.137845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:45.156475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812a70-c9f8-4332-aaff-34e30a32ee65 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:45.249516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf3c80-d3b9-47c0-b98c-89fc2fb57e61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:45.251354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3c80-d3b9-47c0-b98c-89fc2fb57e61 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:45.252116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3c80-d3b9-47c0-b98c-89fc2fb57e61 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:45.253604 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf3c80-d3b9-47c0-b98c-89fc2fb57e61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:45.259241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0acf3c80-d3b9-47c0-b98c-89fc2fb57e61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:45.372286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf3c80-d3b9-47c0-b98c-89fc2fb57e61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:45.374244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf3c80-d3b9-47c0-b98c-89fc2fb57e61 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:45.385645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1108/2228] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:46.103047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:46.105225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:46.174326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcc34d-b0a5-47e4-9bb7-29cb9f658234 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:46.180108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:46.181031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:46.181313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:46.181591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] PATH_INFO: `/v3/users/1f3244c979ba4c72a21b6c7426fcc8d8` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:46.182803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3244c979ba4c72a21b6c7426fcc8d8)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:46.184818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:46.246047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3244c979ba4c72a21b6c7426fcc8d8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:46.258804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3244c979ba4c72a21b6c7426fcc8d8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:46.259676 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3244c979ba4c72a21b6c7426fcc8d8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:46.260889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3244c979ba4c72a21b6c7426fcc8d8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:46.275694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3244c979ba4c72a21b6c7426fcc8d8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:46.288989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1121/2229] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:46 2022] DELETE /identity/v3/users/1f3244c979ba4c72a21b6c7426fcc8d8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:46.293570 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:46.294087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:46.294452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:46.312015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:46.313624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:46.393584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1109/2230] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:46.404638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:46.406521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:46.541001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173d9d2-179b-4378-865d-64de955a4324 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:46.545432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f76fdb-b1d6-4e2e-a290-7e973cabee9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:46.546101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76fdb-b1d6-4e2e-a290-7e973cabee9e service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:46.546386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76fdb-b1d6-4e2e-a290-7e973cabee9e service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:46.546661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f76fdb-b1d6-4e2e-a290-7e973cabee9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:46.557593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f76fdb-b1d6-4e2e-a290-7e973cabee9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:46.559071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f76fdb-b1d6-4e2e-a290-7e973cabee9e service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:46.574144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1122/2231] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:54:47.007505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.009592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:47.032964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e7dbf-ca25-4fe4-b524-5119eff89352 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.059568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.061132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.061682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.062169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] PATH_INFO: `/v3/projects/b3a6e4e824d5437299c2c0fca3e83d66` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.063777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3a6e4e824d5437299c2c0fca3e83d66)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.066068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.144352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3a6e4e824d5437299c2c0fca3e83d66', 'request_id': 'req-d6ed0074-f8a4-4534-936b-bfaf650de4e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:47.152387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3a6e4e824d5437299c2c0fca3e83d66', 'request_id': 'req-d6ed0074-f8a4-4534-936b-bfaf650de4e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:47.161043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3a6e4e824d5437299c2c0fca3e83d66', 'request_id': 'req-d6ed0074-f8a4-4534-936b-bfaf650de4e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:47.176562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] The token cache is being invalidate because project b3a6e4e824d5437299c2c0fca3e83d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:47.176912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:47.177809 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d6ed0074-f8a4-4534-936b-bfaf650de4e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:47.179433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1110/2232] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:47 2022] DELETE /identity/v3/projects/b3a6e4e824d5437299c2c0fca3e83d66 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:47.184949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00453e-f6e1-4c73-b1b7-7b76035f330d None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.186062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00453e-f6e1-4c73-b1b7-7b76035f330d None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.186528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00453e-f6e1-4c73-b1b7-7b76035f330d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.209367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2e00453e-f6e1-4c73-b1b7-7b76035f330d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:47.210604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2e00453e-f6e1-4c73-b1b7-7b76035f330d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:47.281542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1123/2233] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:47.290789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c40ad4-c59a-4efc-a7b1-60f531a64c6f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.291179 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c40ad4-c59a-4efc-a7b1-60f531a64c6f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.291595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c40ad4-c59a-4efc-a7b1-60f531a64c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.318996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-02c40ad4-c59a-4efc-a7b1-60f531a64c6f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:47.320417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-02c40ad4-c59a-4efc-a7b1-60f531a64c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:47.330699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1111/2234] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:47.334964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00453e-f6e1-4c73-b1b7-7b76035f330d None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.345339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00453e-f6e1-4c73-b1b7-7b76035f330d None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.351079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.351736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.352008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.352274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.353023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:47.353327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.354915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.366909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1124/2235] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:47.371399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c40ad4-c59a-4efc-a7b1-60f531a64c6f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.381141 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c40ad4-c59a-4efc-a7b1-60f531a64c6f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.386308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.386960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.387653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.388037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.388733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.390661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.417495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1112/2236] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:47 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:47.422352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.433866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9d3f1-68c2-49aa-92ba-26c08443a95b admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.498763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.498763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.498763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.498763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.499266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.499266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.521294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1125/2237] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:47 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:47.525363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.548532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55679f92-8bd1-4058-977e-2ce07a6b101b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.557345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.558445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.558733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.559075 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.559901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.562487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.581649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1113/2238] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:47.583955 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.596872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057b8a2-58a2-4e19-bc6c-e2dadb67f651 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.602826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc764bb-344c-4c0c-b1f8-abfa0eaec4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.603641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc764bb-344c-4c0c-b1f8-abfa0eaec4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.603930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc764bb-344c-4c0c-b1f8-abfa0eaec4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.604617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc764bb-344c-4c0c-b1f8-abfa0eaec4a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.604978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc764bb-344c-4c0c-b1f8-abfa0eaec4a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.606694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc764bb-344c-4c0c-b1f8-abfa0eaec4a0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.617611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1126/2239] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:47.622161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.645157 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4879d-22f8-40ac-b2d4-7dc575e14585 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.654129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.655336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.655830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.656400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] PATH_INFO: `/v3/projects/8ac61e7821e343a5a738a545233de3b8/users/f410649331bc4f02aa5fb3b8bf1d8cbd/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.682733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac61e7821e343a5a738a545233de3b8, user_id=f410649331bc4f02aa5fb3b8bf1d8cbd, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.685757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.708492 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1114/2240] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:47 2022] PUT /identity/v3/projects/8ac61e7821e343a5a738a545233de3b8/users/f410649331bc4f02aa5fb3b8bf1d8cbd/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:47.720880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbcc90-7dc5-403d-a84d-f0ed617189c9 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.721207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbcc90-7dc5-403d-a84d-f0ed617189c9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.721512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbcc90-7dc5-403d-a84d-f0ed617189c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.752565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1ecbcc90-7dc5-403d-a84d-f0ed617189c9 None None] MFA Rules not processed for user `f410649331bc4f02aa5fb3b8bf1d8cbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:47.753825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1ecbcc90-7dc5-403d-a84d-f0ed617189c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:47.842824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1127/2241] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:47.854697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:47.857626 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:47.939632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de115a4-8e12-401e-b12a-a46d634958c8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:47.944558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:47.945646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:47.946923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:47.947332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:47.966546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:47.968936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:47.984432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1115/2242] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:49.360326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f92505-f78f-43f2-aa0e-400d0ec8c4aa None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:49.360893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f92505-f78f-43f2-aa0e-400d0ec8c4aa None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:49.360893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f92505-f78f-43f2-aa0e-400d0ec8c4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:49.379368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-74f92505-f78f-43f2-aa0e-400d0ec8c4aa None None] MFA Rules not processed for user `aff6489e5a6441509376c0dca551f03c`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:49.380567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-74f92505-f78f-43f2-aa0e-400d0ec8c4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:49.444848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1128/2243] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:49.468722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:49.470316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:49.590523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2d8e3-baba-447c-a932-6bfa34cb40e8 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:49.603181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:49.604269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:49.604555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:49.604851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] PATH_INFO: `/v3/users/ee679bcf98b149db99ad6b191954b4b0` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:49.606538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee679bcf98b149db99ad6b191954b4b0)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:49.608884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:49.710361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee679bcf98b149db99ad6b191954b4b0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:49.711616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee679bcf98b149db99ad6b191954b4b0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:49.713226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee679bcf98b149db99ad6b191954b4b0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:49.724317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee679bcf98b149db99ad6b191954b4b0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:49.744079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee679bcf98b149db99ad6b191954b4b0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:49.765437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1116/2244] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:49 2022] DELETE /identity/v3/users/ee679bcf98b149db99ad6b191954b4b0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:49.770019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:49.770309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:49.770583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:49.801555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:49.802918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:49.880499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1129/2245] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:54:49.890619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:49.892106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:49.962073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051d08b-a681-49a5-aab5-e7e576a49034 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:49.966086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:49.966829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:49.967112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:49.967384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:49.977514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:49.979445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:49.990887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1117/2246] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:50.343397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:50.345230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:50.361491 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d101870-438f-48c8-ae28-16ac8e5b1617 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:50.366530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:50.369988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:50.369988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:50.369988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] PATH_INFO: `/v3/projects/ef4a60c7e89b4c31b273afb63248a7ff` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:50.369988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4a60c7e89b4c31b273afb63248a7ff)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:50.372101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:50.424694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4a60c7e89b4c31b273afb63248a7ff', 'request_id': 'req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.432755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4a60c7e89b4c31b273afb63248a7ff', 'request_id': 'req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.438797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4a60c7e89b4c31b273afb63248a7ff', 'request_id': 'req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.451159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] The token cache is being invalidate because project ef4a60c7e89b4c31b273afb63248a7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:50.451440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.452265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.454235 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1130/2247] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:50 2022] DELETE /identity/v3/projects/ef4a60c7e89b4c31b273afb63248a7ff => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:50.458034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:50.467956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0be75-df7c-435b-8394-d09ee2411abb service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:50.525741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:50.526633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:50.526909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:50.527184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] PATH_INFO: `/v3/users/aea8ab77da0146fcadde15d08e901f2e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:50.528419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aea8ab77da0146fcadde15d08e901f2e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:50.530398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:50.579788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aea8ab77da0146fcadde15d08e901f2e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.580446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aea8ab77da0146fcadde15d08e901f2e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.581199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aea8ab77da0146fcadde15d08e901f2e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.588226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aea8ab77da0146fcadde15d08e901f2e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.602332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aea8ab77da0146fcadde15d08e901f2e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:50.616360 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1118/2248] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:50 2022] DELETE /identity/v3/users/aea8ab77da0146fcadde15d08e901f2e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:51.287435 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:51.290559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:51.328041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8c9e-8526-4d5d-b4bc-1b4f8e28de8c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:51.389071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:51.389952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:51.390404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:51.390681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] PATH_INFO: `/v3/projects/4f7f6e5e499745bc99307a032d37f159` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:51.392105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f7f6e5e499745bc99307a032d37f159)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:51.394214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:51.443261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f7f6e5e499745bc99307a032d37f159', 'request_id': 'req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.455472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f7f6e5e499745bc99307a032d37f159', 'request_id': 'req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.461968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f7f6e5e499745bc99307a032d37f159', 'request_id': 'req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.474076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] The token cache is being invalidate because project 4f7f6e5e499745bc99307a032d37f159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:51.474367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.475067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.476634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1131/2249] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:51 2022] DELETE /identity/v3/projects/4f7f6e5e499745bc99307a032d37f159 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:51.480618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:51.482112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:51.495713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20873702-4dfc-41e8-a0d2-fda97ea91d60 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:51.557566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:51.558474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:51.558754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:51.559108 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] PATH_INFO: `/v3/users/f5c68af9d89543539fc1e32f321c1c8b` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:51.560497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c68af9d89543539fc1e32f321c1c8b)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:51.563347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:51.616358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c68af9d89543539fc1e32f321c1c8b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.617063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c68af9d89543539fc1e32f321c1c8b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.617686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c68af9d89543539fc1e32f321c1c8b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.624737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c68af9d89543539fc1e32f321c1c8b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.640145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-4f202f62-5827-42f0-902a-57fd5c23ce50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c68af9d89543539fc1e32f321c1c8b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:51.658772 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1119/2250] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:51 2022] DELETE /identity/v3/users/f5c68af9d89543539fc1e32f321c1c8b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:51.956830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:51.958625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:51.971252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ccb88-deb9-4b52-b765-f4eebc612e5f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.041761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.042771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.043052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.043329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] PATH_INFO: `/v3/projects/63ac0280ead94c46a3ee42fe1ec01cca` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.044666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63ac0280ead94c46a3ee42fe1ec01cca)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.046617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.093425 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63ac0280ead94c46a3ee42fe1ec01cca', 'request_id': 'req-476b39f1-0aa8-48b5-96f9-a1236012d8b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:52.101915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63ac0280ead94c46a3ee42fe1ec01cca', 'request_id': 'req-476b39f1-0aa8-48b5-96f9-a1236012d8b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:52.107434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63ac0280ead94c46a3ee42fe1ec01cca', 'request_id': 'req-476b39f1-0aa8-48b5-96f9-a1236012d8b6'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:52.118648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] The token cache is being invalidate because project 63ac0280ead94c46a3ee42fe1ec01cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:52.118937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:52.119621 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-476b39f1-0aa8-48b5-96f9-a1236012d8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:52.121469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1132/2251] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:51 2022] DELETE /identity/v3/projects/63ac0280ead94c46a3ee42fe1ec01cca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:52.129364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ff863-8c5e-48d6-85dc-6a9797ffb877 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.129828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ff863-8c5e-48d6-85dc-6a9797ffb877 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.130219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ff863-8c5e-48d6-85dc-6a9797ffb877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.151963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-934ff863-8c5e-48d6-85dc-6a9797ffb877 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:52.153280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-934ff863-8c5e-48d6-85dc-6a9797ffb877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:52.220120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1120/2252] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:52.234456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfec0c-f064-496f-8ed3-85f0ec753e34 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.234694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfec0c-f064-496f-8ed3-85f0ec753e34 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.234997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfec0c-f064-496f-8ed3-85f0ec753e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.262776 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0dbfec0c-f064-496f-8ed3-85f0ec753e34 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:52.264682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0dbfec0c-f064-496f-8ed3-85f0ec753e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:52.275222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1133/2253] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:52.279867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ff863-8c5e-48d6-85dc-6a9797ffb877 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:52.289456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ff863-8c5e-48d6-85dc-6a9797ffb877 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.294710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.295382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.295556 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.296030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.296763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:52.297076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.298831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.316719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1121/2254] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:52.321161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfec0c-f064-496f-8ed3-85f0ec753e34 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:52.331091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfec0c-f064-496f-8ed3-85f0ec753e34 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.336041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.336834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.337112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.337373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.338065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.339949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.368814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1134/2255] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:52 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:52.373056 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:52.383688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d4533-3eb0-41fa-998a-9a3f961a3441 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.446194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.447069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.447365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.447669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.448433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.450594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.475465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1122/2256] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:52 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:52.481445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:52.492294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e9ce7-2db3-4554-8b1b-3820ed967bf3 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.498030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.498709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.499149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.499351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.500119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.502020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.514301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1135/2257] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:52.520089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:52.534674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c201ca9-0e52-4735-b19d-4b8dc6086231 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.539964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840216ad-690e-40b9-950a-4b4a8ffca548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.540866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840216ad-690e-40b9-950a-4b4a8ffca548 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.541150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840216ad-690e-40b9-950a-4b4a8ffca548 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.541427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840216ad-690e-40b9-950a-4b4a8ffca548 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.542246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840216ad-690e-40b9-950a-4b4a8ffca548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.544268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840216ad-690e-40b9-950a-4b4a8ffca548 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.557687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1123/2258] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:52.564103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:52.576467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d0e7-3720-4648-962a-5489659a55ca admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.581595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.582519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.582800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.583070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] PATH_INFO: `/v3/projects/da7a630cbd2f476a81d60db712b4dbd3/users/3075413bbcba4104bb5fbb907ab2558a/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.600936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7a630cbd2f476a81d60db712b4dbd3, user_id=3075413bbcba4104bb5fbb907ab2558a, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.603590 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.617454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1136/2259] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:52 2022] PUT /identity/v3/projects/da7a630cbd2f476a81d60db712b4dbd3/users/3075413bbcba4104bb5fbb907ab2558a/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:52.622520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.622886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.623227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.641245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:52.642457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:52.711639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1124/2260] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:52.722000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:52.724136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:52.783550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6d7c0-c237-4565-b027-00184ca137bb admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:52.787198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e19f-f9b8-43a1-8d0c-4edae2a8da3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:52.787844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e19f-f9b8-43a1-8d0c-4edae2a8da3c service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:52.788123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e19f-f9b8-43a1-8d0c-4edae2a8da3c service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:52.788390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e19f-f9b8-43a1-8d0c-4edae2a8da3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:52.800934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e19f-f9b8-43a1-8d0c-4edae2a8da3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:52.802681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e19f-f9b8-43a1-8d0c-4edae2a8da3c service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:52.813500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1137/2261] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:53.518782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:53.520317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:53.533500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470e2f8-cea0-4e4f-ab99-b75461d62a1b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:53.542578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:53.544992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:53.545508 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:53.545989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] PATH_INFO: `/v3/users/f410649331bc4f02aa5fb3b8bf1d8cbd` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:53.547641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] RBAC: Authorizing `identity:delete_user(user_id=f410649331bc4f02aa5fb3b8bf1d8cbd)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:53.550598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:53.647754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f410649331bc4f02aa5fb3b8bf1d8cbd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:53.648441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f410649331bc4f02aa5fb3b8bf1d8cbd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:53.649231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f410649331bc4f02aa5fb3b8bf1d8cbd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:53.657333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f410649331bc4f02aa5fb3b8bf1d8cbd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:53.674294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a0ea8269-15f3-40dd-ab84-200985728ced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f410649331bc4f02aa5fb3b8bf1d8cbd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:53.681150 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:53.682967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:53.683919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:53.697629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1125/2262] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:54:53 2022] DELETE /identity/v3/users/f410649331bc4f02aa5fb3b8bf1d8cbd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:53.699528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:53.700089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:53.701098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:53.720196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:53.721817 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:53.721961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:53.723237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:53.804605 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1126/2263] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:53.821563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:53.821563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:53.827697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1138/2264] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:54:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:53.896518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:53.899055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:53.911836 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77540af-fdd2-48f6-acf5-1f2961d00aaa None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:53.918139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:53.918939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:53.919456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:53.919796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:53.934509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:53.936100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:53.949294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1127/2265] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:54:54.074205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8e7f-16f6-487f-b262-46c1c30440ed None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:54.081440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe3b4-34ce-43e3-86c6-49f573824fdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:54.082470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe3b4-34ce-43e3-86c6-49f573824fdf service swift] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:54.082961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe3b4-34ce-43e3-86c6-49f573824fdf service swift] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:54.083372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe3b4-34ce-43e3-86c6-49f573824fdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:54.096876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe3b4-34ce-43e3-86c6-49f573824fdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:54.098733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe3b4-34ce-43e3-86c6-49f573824fdf service swift] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:54.115386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1139/2266] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:54:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:54:54.302224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:54.305744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:54.361576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c8608-0c2a-43b7-8409-7c49084c01a6 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:54.378101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:54.378958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:54.379400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:54.379823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] PATH_INFO: `/v3/projects/8ac61e7821e343a5a738a545233de3b8` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:54.381781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac61e7821e343a5a738a545233de3b8)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:54.383884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:54.545057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac61e7821e343a5a738a545233de3b8', 'request_id': 'req-49598327-0a87-4164-b84b-31c5c0e7e54e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:54.561830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac61e7821e343a5a738a545233de3b8', 'request_id': 'req-49598327-0a87-4164-b84b-31c5c0e7e54e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:54.578471 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac61e7821e343a5a738a545233de3b8', 'request_id': 'req-49598327-0a87-4164-b84b-31c5c0e7e54e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:54.609740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] The token cache is being invalidate because project 8ac61e7821e343a5a738a545233de3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:54:54.610024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:54.611039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-49598327-0a87-4164-b84b-31c5c0e7e54e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:54:54.613721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1128/2267] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:54:54 2022] DELETE /identity/v3/projects/8ac61e7821e343a5a738a545233de3b8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:54.618913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05137a7-aa18-4b37-8177-547586724aba None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:54.619198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05137a7-aa18-4b37-8177-547586724aba None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:54.619497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05137a7-aa18-4b37-8177-547586724aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:54.659983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f05137a7-aa18-4b37-8177-547586724aba None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:54.662401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f05137a7-aa18-4b37-8177-547586724aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:54.753632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e494c7c-c8d0-47a6-8424-4be296f588bd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:54.755011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e494c7c-c8d0-47a6-8424-4be296f588bd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:54.755506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e494c7c-c8d0-47a6-8424-4be296f588bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:54.759842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1140/2268] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:54.783911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6e494c7c-c8d0-47a6-8424-4be296f588bd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:54.786052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6e494c7c-c8d0-47a6-8424-4be296f588bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:54.804220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05137a7-aa18-4b37-8177-547586724aba None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:54.812738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1129/2269] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:54.817740 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05137a7-aa18-4b37-8177-547586724aba None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:54.849166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:54.851070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:54.851792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:54.852807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:54.856880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:54:54.856880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:54.861622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:54.895940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e494c7c-c8d0-47a6-8424-4be296f588bd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:54.907108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1141/2270] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:54:54.908255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e494c7c-c8d0-47a6-8424-4be296f588bd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:54.914878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:54.916101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:54.917097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:54.917603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:54.918698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:54.921958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:54.969944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:54.976544 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1130/2271] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:54 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:54.986447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c985d6d-04b8-412a-a72f-15d917c13fe9 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:55.278373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:55.279810 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:55.280239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:55.280668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:55.282701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:55.284922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:55.361736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1142/2272] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:54 2022] POST /identity/v3/users => generated 289 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:55.364909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:55.377874 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2a81d-e9fd-45f9-ae61-7c61f2488111 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:55.398803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:55.401496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:55.401731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:55.402419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:55.405520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:55.407986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:55.447517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1131/2273] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:55.462373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:55.481834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8001d4-c64d-4d03-a15a-ef8755992d70 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:55.490493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f945e87-5332-410c-ae0e-432b07dc7d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:55.491976 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f945e87-5332-410c-ae0e-432b07dc7d09 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:55.492447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f945e87-5332-410c-ae0e-432b07dc7d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:55.493069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f945e87-5332-410c-ae0e-432b07dc7d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:55.494312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f945e87-5332-410c-ae0e-432b07dc7d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:55.496296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f945e87-5332-410c-ae0e-432b07dc7d09 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:55.528932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1143/2274] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:55.533140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:55.553262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac845c0a-ae58-4d3b-a6e9-826101955c3d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:55.573615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:55.574470 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:55.574964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:55.577088 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] PATH_INFO: `/v3/projects/824b8f458d15486c97a42fb7c678199a/users/b10caa9bd3ff4d29aa81a581451d7c44/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:55.623995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=824b8f458d15486c97a42fb7c678199a, user_id=b10caa9bd3ff4d29aa81a581451d7c44, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:55.627295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:55.651172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1132/2275] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:55 2022] PUT /identity/v3/projects/824b8f458d15486c97a42fb7c678199a/users/b10caa9bd3ff4d29aa81a581451d7c44/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:55.661495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38e69c-d3a7-4817-9162-929518f6f0d0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:55.661874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38e69c-d3a7-4817-9162-929518f6f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:55.662104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38e69c-d3a7-4817-9162-929518f6f0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:55.741371 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-db38e69c-d3a7-4817-9162-929518f6f0d0 None None] MFA Rules not processed for user `b10caa9bd3ff4d29aa81a581451d7c44`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:55.774515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-db38e69c-d3a7-4817-9162-929518f6f0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:56.039445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1144/2276] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:56.043072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:56.265621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063f5ba-8336-4882-a8c0-bd5c32b35f83 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:56.282930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedb48d-0c91-4b62-ad1a-eb107a64ffbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:56.284636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedb48d-0c91-4b62-ad1a-eb107a64ffbf admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:56.284960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedb48d-0c91-4b62-ad1a-eb107a64ffbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:56.285240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedb48d-0c91-4b62-ad1a-eb107a64ffbf admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:56.290386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedb48d-0c91-4b62-ad1a-eb107a64ffbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:56.304761 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedb48d-0c91-4b62-ad1a-eb107a64ffbf admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:56.392428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1133/2277] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:56 2022] POST /identity/v3/projects => generated 377 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:56.396501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38e69c-d3a7-4817-9162-929518f6f0d0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:56.443112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38e69c-d3a7-4817-9162-929518f6f0d0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:56.510395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb499e-55b0-43fa-87af-f52f1ca388d8 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:56.510974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb499e-55b0-43fa-87af-f52f1ca388d8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:56.513789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb499e-55b0-43fa-87af-f52f1ca388d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:56.579242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-fafb499e-55b0-43fa-87af-f52f1ca388d8 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:56.581065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fafb499e-55b0-43fa-87af-f52f1ca388d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:56.638398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:56.639211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:56.639739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:56.640091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:56.648920 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:56.652085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:56.691282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1145/2278] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:56 2022] POST /identity/v3/users => generated 289 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:56.697976 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:56.723324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa27343-b5ba-426e-b50f-20dd7158b00a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:56.735136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:56.736353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:56.736779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:56.737240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:56.745446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:56.764752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:56.781562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1134/2279] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:56.801195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1146/2280] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:56.808869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb499e-55b0-43fa-87af-f52f1ca388d8 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:56.834016 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:56.835257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb499e-55b0-43fa-87af-f52f1ca388d8 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:56.838459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:56.857993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:56.858965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:56.859268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:56.864099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:56.872967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:56.877509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:56.974742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1135/2281] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:56.975735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:57.005826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee0b32-d162-400d-bdd6-750d614eb4cf admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:57.015256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e9332-dac1-4804-8130-9ec2b68902a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:57.016687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e9332-dac1-4804-8130-9ec2b68902a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:57.017981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e9332-dac1-4804-8130-9ec2b68902a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:57.018652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e9332-dac1-4804-8130-9ec2b68902a0 admin admin] PATH_INFO: `/v3/projects/bfe9192e3cb04930bd3dbd1f0967c814/users/5b6f7742a3c94704bfcb45646ed1935f/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:57.026499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395fb51-f22b-46fe-9129-eaee5c78381b admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:57.040585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:57.041591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:57.042411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:57.042411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:57.063278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e9332-dac1-4804-8130-9ec2b68902a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe9192e3cb04930bd3dbd1f0967c814, user_id=5b6f7742a3c94704bfcb45646ed1935f, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:57.066803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:57.067457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e9332-dac1-4804-8130-9ec2b68902a0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:57.068450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:57.089549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1136/2282] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:56 2022] PUT /identity/v3/projects/bfe9192e3cb04930bd3dbd1f0967c814/users/5b6f7742a3c94704bfcb45646ed1935f/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:57.091451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1147/2283] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:54:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:54:57.102714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b10e7-b27a-4336-aa3b-a5ec3d3a4cc7 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:57.103491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b10e7-b27a-4336-aa3b-a5ec3d3a4cc7 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:57.104077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b10e7-b27a-4336-aa3b-a5ec3d3a4cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:57.166607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5a7b10e7-b27a-4336-aa3b-a5ec3d3a4cc7 None None] MFA Rules not processed for user `5b6f7742a3c94704bfcb45646ed1935f`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:57.167861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5a7b10e7-b27a-4336-aa3b-a5ec3d3a4cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:57.409609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1137/2284] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:57.416628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:57.436487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67324c-7d0c-4ab6-91bb-f721ec27f855 service swift] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:57.623123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:57.624121 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:57.624580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:57.625043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:57.625941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:57.628085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:57.687505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1148/2285] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:54:57 2022] POST /identity/v3/projects => generated 377 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:57.691235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b10e7-b27a-4336-aa3b-a5ec3d3a4cc7 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:57.719841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b10e7-b27a-4336-aa3b-a5ec3d3a4cc7 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:58.017779 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:58.018579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:58.018907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:58.019192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:58.019930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:58.021977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:58.107283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1138/2286] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:54:57 2022] POST /identity/v3/users => generated 295 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:54:58.108230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:58.124193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840387d2-fe4c-43a5-b761-7dd3a0670b5d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:58.163034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:58.164089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:58.164576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:58.165041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:58.166176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:58.168136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:58.211758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:58.212013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1149/2287] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:54:58.233631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c65b15-9ba6-4393-ac07-e0cedc5e70e8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:58.272994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:58.274005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:58.274300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:58.274603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] PATH_INFO: `/v3/projects/7ff6d985f739421e9b95521aa63a1a5e/users/7c3c477f3cbe4ac0b51cc9f13e245525/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:58.347465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff6d985f739421e9b95521aa63a1a5e, user_id=7c3c477f3cbe4ac0b51cc9f13e245525, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:58.368542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:58.405159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1139/2288] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:58 2022] PUT /identity/v3/projects/7ff6d985f739421e9b95521aa63a1a5e/users/7c3c477f3cbe4ac0b51cc9f13e245525/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:54:58.409232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:58.432196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e694e-70f0-4ab4-b849-d356f3f1fc6e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:58.683165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce49710-268a-4fa1-863b-c7dad4ec9d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:58.683930 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce49710-268a-4fa1-863b-c7dad4ec9d62 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:58.684233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce49710-268a-4fa1-863b-c7dad4ec9d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:58.684516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce49710-268a-4fa1-863b-c7dad4ec9d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:58.687124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce49710-268a-4fa1-863b-c7dad4ec9d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:58.690800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce49710-268a-4fa1-863b-c7dad4ec9d62 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:58.720068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1150/2289] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:54:58.721395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:58.774623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4679e-e865-492b-98e5-13ffe4428522 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:58.803927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:58.805321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:58.805802 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:58.806254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] PATH_INFO: `/v3/projects/7ff6d985f739421e9b95521aa63a1a5e/users/7c3c477f3cbe4ac0b51cc9f13e245525/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:58.811476 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff6d985f739421e9b95521aa63a1a5e, user_id=7c3c477f3cbe4ac0b51cc9f13e245525, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:58.816395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:58.853286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1140/2290] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:54:58 2022] PUT /identity/v3/projects/7ff6d985f739421e9b95521aa63a1a5e/users/7c3c477f3cbe4ac0b51cc9f13e245525/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:54:58.864041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:58.864400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:58.864775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:59.011433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] MFA Rules not processed for user `7c3c477f3cbe4ac0b51cc9f13e245525`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:54:59.018105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:59.191666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1151/2291] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:54:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:54:59.213178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:54:59.215877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:54:59.391637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b611d-4080-408c-b05b-521ec93cce39 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:54:59.400339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:54:59.401291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:54:59.401752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:54:59.404073 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:54:59.554663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:54:59.555968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:54:59.598878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1141/2292] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:54:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:01.959070 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:01.960664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:02.009357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1e6c4-b33a-4ea2-a923-eb8514b3d632 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:02.054515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654fdfd-39b6-4cf5-b515-80b20c2e89fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:02.055209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654fdfd-39b6-4cf5-b515-80b20c2e89fc service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:02.062023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654fdfd-39b6-4cf5-b515-80b20c2e89fc service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:02.066332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654fdfd-39b6-4cf5-b515-80b20c2e89fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:02.384985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654fdfd-39b6-4cf5-b515-80b20c2e89fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:02.388054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654fdfd-39b6-4cf5-b515-80b20c2e89fc service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:02.440069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1152/2293] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:03.955535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:03.966822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152735b-c4bd-4ee8-83d8-e0edf205d58c service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:03.971883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db4f32-88a5-480a-a1a3-b40915bb7b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:03.972568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db4f32-88a5-480a-a1a3-b40915bb7b6f service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:03.975463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db4f32-88a5-480a-a1a3-b40915bb7b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:03.975819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db4f32-88a5-480a-a1a3-b40915bb7b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:04.001482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db4f32-88a5-480a-a1a3-b40915bb7b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:04.009938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db4f32-88a5-480a-a1a3-b40915bb7b6f service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:04.066932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1142/2294] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:04.908417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:04.908987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:04.908987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:04.948059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] MFA Rules not processed for user `06d696a14671493db2c243300a19cbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:04.953036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:05.110447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1153/2295] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:05.986188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7161e59-fe2e-4006-9171-ed8faa9836b1 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:05.986783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7161e59-fe2e-4006-9171-ed8faa9836b1 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:05.986947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7161e59-fe2e-4006-9171-ed8faa9836b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:06.114146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a7161e59-fe2e-4006-9171-ed8faa9836b1 None None] MFA Rules not processed for user `3075413bbcba4104bb5fbb907ab2558a`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:06.115648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a7161e59-fe2e-4006-9171-ed8faa9836b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:06.359959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:06.360889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1143/2296] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:55:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:06.362172 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:06.483112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2db79-5fe5-45f0-9b6f-973a352d0a5a None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:06.488401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:06.489428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:06.489900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:06.490352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:06.491303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:06.493847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:06.529316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1154/2297] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:55:06 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:55:06.533627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7161e59-fe2e-4006-9171-ed8faa9836b1 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:06.550937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7161e59-fe2e-4006-9171-ed8faa9836b1 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:06.701833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:06.702898 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:06.707212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:06.707741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:06.708703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:06.714009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:06.769193 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:06.770392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1144/2298] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:55:06 2022] POST /identity/v3/users => generated 295 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:55:06.786059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65120d-b43c-4ec6-a512-a59d0ac8105b admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:06.798799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:06.799828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:06.804069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:06.804708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:06.808359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:06.810388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:06.836102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:06.836560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1155/2299] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:55:06.849337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75082-333d-4eca-bf17-42e16dbd5d29 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:06.868147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:06.869107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:06.869386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:06.869664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] PATH_INFO: `/v3/projects/ed26aba538f0415fa3fe812554c97365/users/cc91883c9d15451fabe899531d318a1f/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:06.920219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed26aba538f0415fa3fe812554c97365, user_id=cc91883c9d15451fabe899531d318a1f, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:06.922968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:06.955210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1145/2300] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:55:06 2022] PUT /identity/v3/projects/ed26aba538f0415fa3fe812554c97365/users/cc91883c9d15451fabe899531d318a1f/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:55:06.957488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:06.976159 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b33e5f-51ce-4978-a980-a5958591a52c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:07.061555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:07.062720 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:07.063385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:07.063980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:07.065019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:07.066985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:07.082113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1156/2301] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:55:07.086620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:07.104763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bbbfb-7582-4d5e-abec-f5ad9b76d2a5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:07.117166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa5a90-33d6-43e0-89ac-6de59b874ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:07.118538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5a90-33d6-43e0-89ac-6de59b874ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:07.119105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5a90-33d6-43e0-89ac-6de59b874ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:07.119620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5a90-33d6-43e0-89ac-6de59b874ea3 admin admin] PATH_INFO: `/v3/projects/ed26aba538f0415fa3fe812554c97365/users/cc91883c9d15451fabe899531d318a1f/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:07.126282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa5a90-33d6-43e0-89ac-6de59b874ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed26aba538f0415fa3fe812554c97365, user_id=cc91883c9d15451fabe899531d318a1f, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:07.129432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa5a90-33d6-43e0-89ac-6de59b874ea3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:07.146566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1146/2302] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:55:07 2022] PUT /identity/v3/projects/ed26aba538f0415fa3fe812554c97365/users/cc91883c9d15451fabe899531d318a1f/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:55:09.340098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:09.351101 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c74c3-0fec-498f-b67e-6dbabbb3b14e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:09.565467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:09.566497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:09.566896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:09.567273 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] PATH_INFO: `/v3/users/b10caa9bd3ff4d29aa81a581451d7c44` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:09.569038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b10caa9bd3ff4d29aa81a581451d7c44)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:09.571918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:09.729245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b10caa9bd3ff4d29aa81a581451d7c44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:09.788040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b10caa9bd3ff4d29aa81a581451d7c44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:09.791108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b10caa9bd3ff4d29aa81a581451d7c44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:09.793404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b10caa9bd3ff4d29aa81a581451d7c44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:09.818837 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b10caa9bd3ff4d29aa81a581451d7c44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:09.872788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1157/2303] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:55:09 2022] DELETE /identity/v3/users/b10caa9bd3ff4d29aa81a581451d7c44 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:55:09.878317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:09.878816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:09.878917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:09.932147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:09.933515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:10.162687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1147/2304] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:55:10.194643 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:10.205854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:10.407496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee4f77-6649-4c04-968f-786b32a88913 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:10.418909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:10.419922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:10.420362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:10.422000 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:10.447699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:10.452224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:10.478829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1158/2305] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:11.395374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:11.397003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:11.439904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498557cf-aea4-4da0-9201-df5d5604266b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:11.458407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:11.461485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:11.463193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:11.464435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] PATH_INFO: `/v3/projects/824b8f458d15486c97a42fb7c678199a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:11.469633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=824b8f458d15486c97a42fb7c678199a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:11.477456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:11.588417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '824b8f458d15486c97a42fb7c678199a', 'request_id': 'req-389e74a9-65e4-452e-87bc-3ba79131bb52'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:11.615554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '824b8f458d15486c97a42fb7c678199a', 'request_id': 'req-389e74a9-65e4-452e-87bc-3ba79131bb52'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:11.645391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '824b8f458d15486c97a42fb7c678199a', 'request_id': 'req-389e74a9-65e4-452e-87bc-3ba79131bb52'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:11.672116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] The token cache is being invalidate because project 824b8f458d15486c97a42fb7c678199a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:55:11.675005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:11.675881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:11.684100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1148/2306] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:55:11 2022] DELETE /identity/v3/projects/824b8f458d15486c97a42fb7c678199a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:55:11.688392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:11.702816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b7f80-94ec-4b1e-879d-6fbeef756cb3 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:11.874114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:11.876680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:11.880568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:11.881381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] PATH_INFO: `/v3/users/5b6f7742a3c94704bfcb45646ed1935f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:11.883562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6f7742a3c94704bfcb45646ed1935f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:11.897515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:12.053889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6f7742a3c94704bfcb45646ed1935f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:12.082958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6f7742a3c94704bfcb45646ed1935f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:12.083559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6f7742a3c94704bfcb45646ed1935f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:12.086106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6f7742a3c94704bfcb45646ed1935f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:12.104224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6f7742a3c94704bfcb45646ed1935f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:12.140141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1159/2307] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:55:11 2022] DELETE /identity/v3/users/5b6f7742a3c94704bfcb45646ed1935f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:55:12.831448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:12.833031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:12.874459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e74a9-65e4-452e-87bc-3ba79131bb52 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:13.086841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:13.087941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:13.088226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:13.088503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] PATH_INFO: `/v3/projects/bfe9192e3cb04930bd3dbd1f0967c814` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:13.091579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe9192e3cb04930bd3dbd1f0967c814)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:13.093553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:13.271397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe9192e3cb04930bd3dbd1f0967c814', 'request_id': 'req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.293026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe9192e3cb04930bd3dbd1f0967c814', 'request_id': 'req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.312612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe9192e3cb04930bd3dbd1f0967c814', 'request_id': 'req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.349612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] The token cache is being invalidate because project bfe9192e3cb04930bd3dbd1f0967c814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:55:13.352674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.353627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.364993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:13.366494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:13.377436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1149/2308] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:55:12 2022] DELETE /identity/v3/projects/bfe9192e3cb04930bd3dbd1f0967c814 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:55:13.400071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855c16b-d6a2-4bdb-8fdb-f1acc4cc45bb admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:13.674007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:13.674937 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:13.675218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:13.675490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] PATH_INFO: `/v3/users/7c3c477f3cbe4ac0b51cc9f13e245525` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:13.677834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3c477f3cbe4ac0b51cc9f13e245525)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:13.684281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:13.847884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3c477f3cbe4ac0b51cc9f13e245525'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.884020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3c477f3cbe4ac0b51cc9f13e245525'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.884944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3c477f3cbe4ac0b51cc9f13e245525'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.885801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3c477f3cbe4ac0b51cc9f13e245525'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.911939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-4c6167bc-d206-4890-8290-b56ca7907fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3c477f3cbe4ac0b51cc9f13e245525'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:13.929878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1160/2309] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:55:13 2022] DELETE /identity/v3/users/7c3c477f3cbe4ac0b51cc9f13e245525 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:55:14.630473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:14.632205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:14.657144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541f3e4-6d70-4ebb-99b4-6f6fda2117e2 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:14.851713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:14.853030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:14.853511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:14.853957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] PATH_INFO: `/v3/projects/7ff6d985f739421e9b95521aa63a1a5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:14.860960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff6d985f739421e9b95521aa63a1a5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:14.864554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:15.017583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff6d985f739421e9b95521aa63a1a5e', 'request_id': 'req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:15.033624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff6d985f739421e9b95521aa63a1a5e', 'request_id': 'req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:15.049185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff6d985f739421e9b95521aa63a1a5e', 'request_id': 'req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:15.079445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] The token cache is being invalidate because project 7ff6d985f739421e9b95521aa63a1a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:55:15.080331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:15.082557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:15.100856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1150/2310] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:55:14 2022] DELETE /identity/v3/projects/7ff6d985f739421e9b95521aa63a1a5e => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:55:19.781378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:19.782336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:19.782617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:19.815251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] MFA Rules not processed for user `cc91883c9d15451fabe899531d318a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:19.817800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:19.930906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1161/2311] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:55:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:20.155887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:20.158889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:20.305535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8fcd8-b8f7-4df5-b790-fbfe79101260 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:20.311675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adbb56-38f4-4e74-9e5c-824b7e65e5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:20.312364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbb56-38f4-4e74-9e5c-824b7e65e5e9 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:20.312644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbb56-38f4-4e74-9e5c-824b7e65e5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:20.313004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbb56-38f4-4e74-9e5c-824b7e65e5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:20.317545 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-18adbb56-38f4-4e74-9e5c-824b7e65e5e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:20.404866 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adbb56-38f4-4e74-9e5c-824b7e65e5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:20.406733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adbb56-38f4-4e74-9e5c-824b7e65e5e9 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:20.423554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1151/2312] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:21.296572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:21.298313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:21.576041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c21740-63f3-465f-88da-e35259b997c7 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:21.588643 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157a373-5d0e-49ce-96b8-af55f254360f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac21bca8da642338571f821ef5ea121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:21.591049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a373-5d0e-49ce-96b8-af55f254360f service placement] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:21.593064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a373-5d0e-49ce-96b8-af55f254360f service placement] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:21.593814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a373-5d0e-49ce-96b8-af55f254360f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:21.600076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e157a373-5d0e-49ce-96b8-af55f254360f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:21.628496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157a373-5d0e-49ce-96b8-af55f254360f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:21.630233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157a373-5d0e-49ce-96b8-af55f254360f service placement] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:21.667007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1162/2313] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:27.561151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8a9d-de0b-49fc-ad36-b475c8c39676 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:27.561782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8a9d-de0b-49fc-ad36-b475c8c39676 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:27.561782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8a9d-de0b-49fc-ad36-b475c8c39676 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:27.563150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1152/2314] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:55:27.569703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8813f7-94d6-46c5-bab4-5b42b4ecac52 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:27.570247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8813f7-94d6-46c5-bab4-5b42b4ecac52 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:27.570472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8813f7-94d6-46c5-bab4-5b42b4ecac52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:27.614899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-bb8813f7-94d6-46c5-bab4-5b42b4ecac52 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:27.616157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bb8813f7-94d6-46c5-bab4-5b42b4ecac52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:27.626182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1163/2315] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:27.638393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb8a9d-de0b-49fc-ad36-b475c8c39676 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:27.640302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b3bb8a9d-de0b-49fc-ad36-b475c8c39676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:27.765093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb8a9d-de0b-49fc-ad36-b475c8c39676 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:27.768893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83db638-5886-46a6-a031-3141d4a252c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:27.769543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83db638-5886-46a6-a031-3141d4a252c9 service glance] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:27.769853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83db638-5886-46a6-a031-3141d4a252c9 service glance] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:27.770101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83db638-5886-46a6-a031-3141d4a252c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:27.780401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83db638-5886-46a6-a031-3141d4a252c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:27.781932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83db638-5886-46a6-a031-3141d4a252c9 service glance] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:27.792358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1153/2316] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:27.866496 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283337f-1ae6-439c-abf7-ae2b9833258f None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:27.866784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283337f-1ae6-439c-abf7-ae2b9833258f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:27.867230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283337f-1ae6-439c-abf7-ae2b9833258f None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:27.868759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1164/2317] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:55:27.874346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567f91-7f70-46bf-9930-005879598ed6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:27.874641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567f91-7f70-46bf-9930-005879598ed6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:27.874944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567f91-7f70-46bf-9930-005879598ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:27.895889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-17567f91-7f70-46bf-9930-005879598ed6 None None] MFA Rules not processed for user `0d678d5ee7d74896bbc1316b32ec81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:27.897322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-17567f91-7f70-46bf-9930-005879598ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:27.959014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1154/2318] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:27.971870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283337f-1ae6-439c-abf7-ae2b9833258f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:27.973820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b283337f-1ae6-439c-abf7-ae2b9833258f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:27.987932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283337f-1ae6-439c-abf7-ae2b9833258f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:27.992490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c69a3b-1527-428f-9e97-63be9ecf82b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:27.993220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c69a3b-1527-428f-9e97-63be9ecf82b6 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:27.993507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c69a3b-1527-428f-9e97-63be9ecf82b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:27.993782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c69a3b-1527-428f-9e97-63be9ecf82b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:28.003454 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c69a3b-1527-428f-9e97-63be9ecf82b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:28.003693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c69a3b-1527-428f-9e97-63be9ecf82b6 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:28.015107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1165/2319] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:28.111478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956e6f-ae0a-4ec1-8c81-48336e74a1b4 None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:28.111985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956e6f-ae0a-4ec1-8c81-48336e74a1b4 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:28.112367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956e6f-ae0a-4ec1-8c81-48336e74a1b4 None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:28.114566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1155/2320] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:55:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:55:28.121811 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e26051-9c6b-4920-8ed8-f77b5ff72e0a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:28.122623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e26051-9c6b-4920-8ed8-f77b5ff72e0a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:28.123381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e26051-9c6b-4920-8ed8-f77b5ff72e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:28.151581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-53e26051-9c6b-4920-8ed8-f77b5ff72e0a None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:28.152823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-53e26051-9c6b-4920-8ed8-f77b5ff72e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:28.164227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1166/2321] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:28.175699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956e6f-ae0a-4ec1-8c81-48336e74a1b4 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:28.177861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2c956e6f-ae0a-4ec1-8c81-48336e74a1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:28.197277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956e6f-ae0a-4ec1-8c81-48336e74a1b4 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:28.201631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:28.202479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:28.202633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:28.202902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:28.213851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:28.215481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:28.226516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1156/2322] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:55:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:55:28.580148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:28.580662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:28.580662 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:28.641099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:28.642481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:28.713783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1167/2323] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 4 14:55:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:28.730005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:28.731512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:28.837478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c0450-1446-4721-8775-cd83f658144f service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:28.842863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:28.843529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:28.843813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:28.844090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:28.853228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:28.854972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:28.863914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1157/2324] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:55:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:55:31.099080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:31.100637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:31.198504 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69ab6d-563e-4eea-b788-13aebb344ac0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:31.206021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:31.207285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:31.207725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:31.208163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] PATH_INFO: `/v3/users/7533ae6ea6774f7eb853455d086f939d` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:31.209549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7533ae6ea6774f7eb853455d086f939d)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:31.211560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:31.285338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7533ae6ea6774f7eb853455d086f939d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:31.301183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7533ae6ea6774f7eb853455d086f939d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:31.303445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7533ae6ea6774f7eb853455d086f939d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:31.304465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7533ae6ea6774f7eb853455d086f939d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:31.316570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-02a8a59d-7104-4eda-99f7-f2de96d6a22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7533ae6ea6774f7eb853455d086f939d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:31.336222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1168/2325] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:55:31 2022] DELETE /identity/v3/users/7533ae6ea6774f7eb853455d086f939d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:55:31.778083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:31.823202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a40c03-fd98-45c9-aa26-16505a18c508 service swift] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:31.911036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:31.912216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:31.912589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:31.913122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] PATH_INFO: `/v3/projects/f2a9a0c14546490db1746dda5aedd3ea` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:31.914460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a9a0c14546490db1746dda5aedd3ea)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:31.917177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:31.980268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a9a0c14546490db1746dda5aedd3ea', 'request_id': 'req-22e2d420-eedb-4a25-9477-514b39a93c28'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:31.990727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a9a0c14546490db1746dda5aedd3ea', 'request_id': 'req-22e2d420-eedb-4a25-9477-514b39a93c28'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:31.997014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a9a0c14546490db1746dda5aedd3ea', 'request_id': 'req-22e2d420-eedb-4a25-9477-514b39a93c28'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:32.013542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] The token cache is being invalidate because project f2a9a0c14546490db1746dda5aedd3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:55:32.013992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:32.014690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:55:32.016322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1158/2326] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:55:31 2022] DELETE /identity/v3/projects/f2a9a0c14546490db1746dda5aedd3ea => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:55:32.830006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca02bcb-a20b-4711-a855-70b9730d3864 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:32.830577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca02bcb-a20b-4711-a855-70b9730d3864 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:32.830675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca02bcb-a20b-4711-a855-70b9730d3864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:32.860314 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8ca02bcb-a20b-4711-a855-70b9730d3864 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:32.861624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8ca02bcb-a20b-4711-a855-70b9730d3864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:32.935206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1169/2327] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:55:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:32.949691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:32.951198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:33.040975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2d420-eedb-4a25-9477-514b39a93c28 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:33.057646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df9e0a-b134-48c0-af12-434b12722a40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:33.058326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df9e0a-b134-48c0-af12-434b12722a40 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:33.058621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df9e0a-b134-48c0-af12-434b12722a40 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:33.059070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df9e0a-b134-48c0-af12-434b12722a40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:33.069485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df9e0a-b134-48c0-af12-434b12722a40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:33.071201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df9e0a-b134-48c0-af12-434b12722a40 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:33.083595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1159/2328] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:55:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:55:42.439045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e78492-092e-4c22-98b2-c97ac195aa69 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:42.439702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e78492-092e-4c22-98b2-c97ac195aa69 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:42.439702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e78492-092e-4c22-98b2-c97ac195aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:42.462284 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a9e78492-092e-4c22-98b2-c97ac195aa69 None None] MFA Rules not processed for user `3075413bbcba4104bb5fbb907ab2558a`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:42.463814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a9e78492-092e-4c22-98b2-c97ac195aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:42.534309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1170/2329] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:55:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:58.293148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:58.293805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:58.293805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:58.314044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:55:58.315409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:55:58.326160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1160/2330] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 4 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:55:58.352882 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e78492-092e-4c22-98b2-c97ac195aa69 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:55:58.365899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e78492-092e-4c22-98b2-c97ac195aa69 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:55:58.371325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d461d-1a2b-4c35-8abc-de8d992c605f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:55:58.371998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d461d-1a2b-4c35-8abc-de8d992c605f service swift] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:55:58.372408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d461d-1a2b-4c35-8abc-de8d992c605f service swift] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:55:58.372711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d461d-1a2b-4c35-8abc-de8d992c605f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:55:58.382137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d461d-1a2b-4c35-8abc-de8d992c605f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:55:58.383771 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d461d-1a2b-4c35-8abc-de8d992c605f service swift] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:55:58.393565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1171/2331] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:55:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:56:02.840475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:56:02.842218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:56:02.902033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b955c4-2722-4b9c-a9ae-45dec9b1f00b None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:56:02.905938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872b41f-9771-4d98-8239-885674521832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:56:02.906648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872b41f-9771-4d98-8239-885674521832 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:56:02.906928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872b41f-9771-4d98-8239-885674521832 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:56:02.907202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872b41f-9771-4d98-8239-885674521832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:56:02.909112 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c872b41f-9771-4d98-8239-885674521832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:56:02.971529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872b41f-9771-4d98-8239-885674521832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:56:02.974135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872b41f-9771-4d98-8239-885674521832 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:56:02.990548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1161/2332] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:56:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:56:36.898952 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30838762-737f-4f39-aeae-9a73f9166805 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:56:36.899489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30838762-737f-4f39-aeae-9a73f9166805 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:56:36.899489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30838762-737f-4f39-aeae-9a73f9166805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:56:36.917931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-30838762-737f-4f39-aeae-9a73f9166805 None None] MFA Rules not processed for user `aff6489e5a6441509376c0dca551f03c`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:56:36.919168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-30838762-737f-4f39-aeae-9a73f9166805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:56:36.982186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1172/2333] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:56:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:56:47.563982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:56:47.564841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:56:47.564841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:56:47.591495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] MFA Rules not processed for user `06d696a14671493db2c243300a19cbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:56:47.592853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:56:47.656242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1162/2334] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:56:51.992756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a695bd-c284-484a-b12d-cb3e6409ce14 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:56:51.993383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a695bd-c284-484a-b12d-cb3e6409ce14 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:56:51.993383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a695bd-c284-484a-b12d-cb3e6409ce14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:56:52.013175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-83a695bd-c284-484a-b12d-cb3e6409ce14 None None] MFA Rules not processed for user `da47bf4c34f243bb9493d8bc11891cef`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:56:52.014396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-83a695bd-c284-484a-b12d-cb3e6409ce14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:56:52.083814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1173/2335] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:56:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:16.936107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:16.937721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:16.953050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73d7d2-b9a2-42b4-a5ab-a4b87de456c8 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:16.958536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aea969-ccf0-446f-a331-e1755c5616dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:16.959210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea969-ccf0-446f-a331-e1755c5616dd service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:16.959609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea969-ccf0-446f-a331-e1755c5616dd service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:16.960095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aea969-ccf0-446f-a331-e1755c5616dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:16.963146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-34aea969-ccf0-446f-a331-e1755c5616dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:17.042045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aea969-ccf0-446f-a331-e1755c5616dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:17.043506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aea969-ccf0-446f-a331-e1755c5616dd service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:17.055956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1163/2336] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:18.244890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a695bd-c284-484a-b12d-cb3e6409ce14 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:18.255450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a695bd-c284-484a-b12d-cb3e6409ce14 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:18.259811 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:18.260457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:18.260737 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:18.261136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:18.263230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:18.277308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:18.278726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:18.290254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1174/2337] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:22.929104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:22.929748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:22.929748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:22.947642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] MFA Rules not processed for user `a0944ce1e8504fb1b50507ace4e63649`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:22.948902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:23.022222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1164/2338] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:57:23.359931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:23.370278 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e680ee-df01-43e9-b4d1-eee6318f9899 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:23.374972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:23.375617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:23.375894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:23.376189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:23.378422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:23.389828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:23.391304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:23.400455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1175/2339] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:23.736370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:23.737853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:23.815558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482f93c-3502-4761-b9f0-550afdcb0750 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:23.824366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac21bca8da642338571f821ef5ea121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:23.825345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:23.825683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:23.826058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:23.828284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:23.840829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:23.842451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:23.852588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1165/2340] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:36.203694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:36.213780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345fdd5-e474-4949-b51b-d2c32ed932c0 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:36.218419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:36.219064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:36.219341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:36.219611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:36.221803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:36.233265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:36.234788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:36.243609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1176/2341] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:39.471050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:39.472622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:39.488759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ef1da-2d37-4456-a1c9-9c0df3b6121a service placement] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:39.495258 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:39.495891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:39.496228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:39.496443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:39.498412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:39.512418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:39.514047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:39.525036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1166/2342] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:43.725322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:43.727004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:43.744487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba01da-5bbd-4f71-b587-4d98db227bbc service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:43.751403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:43.752045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:43.752324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:43.752594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:43.754573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:43.766144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:43.767760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:43.777275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1177/2343] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:43.914098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:43.925271 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab0a27-93f9-42a0-9bfd-d3a7bf225591 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:43.932775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:43.933584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:43.933936 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:43.934212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:43.936251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:43.949194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:43.950939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:43.962006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1167/2344] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:47.993821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:48.006026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeb2ac-5bea-4c39-8ebb-1bb62882d41d service nova] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:48.013449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:48.014110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:48.014394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:48.014699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:48.016907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:48.031065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:48.034168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:48.045019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1178/2345] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:48.695562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:48.706039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de856d9-0bd5-4ce5-b40f-6b3be3ec9dea service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:48.711694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:48.712542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:48.712910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:48.713214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:48.715708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:48.788705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:48.790180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:48.801151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1168/2346] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:54.206859 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:54.208441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:54.223752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91660d9e-71d3-4416-ae9d-897f4de7f29b service nova] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:54.229205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:54.230519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:54.230835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:54.231113 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] PATH_INFO: `/v3/users/06d696a14671493db2c243300a19cbc0` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:54.232683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d696a14671493db2c243300a19cbc0)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:54.234933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:54.299046 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d696a14671493db2c243300a19cbc0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.316168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d696a14671493db2c243300a19cbc0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.321061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d696a14671493db2c243300a19cbc0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.321061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d696a14671493db2c243300a19cbc0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.330041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-23bd7477-90a7-4934-ac27-2d0b4af25e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d696a14671493db2c243300a19cbc0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.349020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1179/2347] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:57:54 2022] DELETE /identity/v3/users/06d696a14671493db2c243300a19cbc0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:54.550773 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:54.562858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eda673-80a1-4375-8ba6-93a96d4a5724 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:54.648241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:54.649167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:54.649447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:54.649739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] PATH_INFO: `/v3/projects/f96454a48c304aabbcb82ad8a2771b01` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:54.651308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96454a48c304aabbcb82ad8a2771b01)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:54.653371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:54.703450 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96454a48c304aabbcb82ad8a2771b01', 'request_id': 'req-429377b9-b01b-4398-b8d6-397b030a93c1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.712139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96454a48c304aabbcb82ad8a2771b01', 'request_id': 'req-429377b9-b01b-4398-b8d6-397b030a93c1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.720078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96454a48c304aabbcb82ad8a2771b01', 'request_id': 'req-429377b9-b01b-4398-b8d6-397b030a93c1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.735600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] The token cache is being invalidate because project f96454a48c304aabbcb82ad8a2771b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:57:54.736048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.736753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-429377b9-b01b-4398-b8d6-397b030a93c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:54.738378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1169/2348] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:57:54 2022] DELETE /identity/v3/projects/f96454a48c304aabbcb82ad8a2771b01 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:54.743909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc27d4-c9d2-4902-b644-c96268e21627 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:54.744198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc27d4-c9d2-4902-b644-c96268e21627 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:54.744786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc27d4-c9d2-4902-b644-c96268e21627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:54.764734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-46cc27d4-c9d2-4902-b644-c96268e21627 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:54.766255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-46cc27d4-c9d2-4902-b644-c96268e21627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:54.844647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1180/2349] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:54.854693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942af0db-297b-49fc-b20f-b29e380bca3d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:54.854984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942af0db-297b-49fc-b20f-b29e380bca3d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:54.855269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942af0db-297b-49fc-b20f-b29e380bca3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:54.879769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-942af0db-297b-49fc-b20f-b29e380bca3d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:54.881188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-942af0db-297b-49fc-b20f-b29e380bca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:54.889759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1170/2350] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:54.895308 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc27d4-c9d2-4902-b644-c96268e21627 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:54.906689 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc27d4-c9d2-4902-b644-c96268e21627 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:54.911791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:54.912528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:54.912834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:54.913115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:54.913870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:57:54.914181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:54.915826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:54.938958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1181/2351] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:57:54.940479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942af0db-297b-49fc-b20f-b29e380bca3d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:54.954158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942af0db-297b-49fc-b20f-b29e380bca3d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:54.959432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:54.960148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:54.960401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:54.960676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:54.961435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:54.963393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:54.995690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1171/2352] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:57:54 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:57:55.000095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.014597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21f774-b4f7-44ba-aafb-20ea74632dcd admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.078198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.078926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.079261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.079631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.080271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.082533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.108370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1182/2353] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:57:54 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:57:55.109895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.121667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ed7b-ce3f-42bb-8d4f-74a37ab9bebf admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.127310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.128125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.128416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.128690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.129524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.131482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.145074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1172/2354] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:55.149675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.160745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12224a5e-1aec-47bf-a345-9a68399b0e25 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.183843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000d4f9-a4df-469c-835d-775116104067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.185106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000d4f9-a4df-469c-835d-775116104067 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.185626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000d4f9-a4df-469c-835d-775116104067 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.186128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000d4f9-a4df-469c-835d-775116104067 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.187559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000d4f9-a4df-469c-835d-775116104067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.190872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000d4f9-a4df-469c-835d-775116104067 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.207044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1183/2355] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:55.211084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.223057 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e231-a35f-42e6-9a11-df0593f8ce00 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.229126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.230095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.230380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.230775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] PATH_INFO: `/v3/projects/3b236511c7434bf1bbfc26eb0f2f7818/users/8999b6906f3e47c697366efafa063288/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.247722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b236511c7434bf1bbfc26eb0f2f7818, user_id=8999b6906f3e47c697366efafa063288, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.250933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.267198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1173/2356] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:57:55 2022] PUT /identity/v3/projects/3b236511c7434bf1bbfc26eb0f2f7818/users/8999b6906f3e47c697366efafa063288/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:57:55.278462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c9bf2-4048-4d0e-93a6-39d437b3e9bb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.278755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c9bf2-4048-4d0e-93a6-39d437b3e9bb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.279032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c9bf2-4048-4d0e-93a6-39d437b3e9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.311857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-eb8c9bf2-4048-4d0e-93a6-39d437b3e9bb None None] MFA Rules not processed for user `8999b6906f3e47c697366efafa063288`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:55.313157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-eb8c9bf2-4048-4d0e-93a6-39d437b3e9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:55.392940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1184/2357] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:55.397878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.467489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28d647-6267-49b6-8788-ff557b5139e5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.473263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.473992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.474269 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.474538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.475278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.477280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.507534 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1174/2358] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:57:55 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:57:55.512663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c9bf2-4048-4d0e-93a6-39d437b3e9bb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.524069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c9bf2-4048-4d0e-93a6-39d437b3e9bb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.586207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.586951 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.587206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.587515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.588220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.590714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.623672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1185/2359] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:57:55 2022] POST /identity/v3/users => generated 308 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:57:55.628940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.640463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801281b-39ed-450a-a49f-365e7f22b006 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.645771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.646461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.646744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.647021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.647771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.653664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.671582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1175/2360] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:55.673001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.685086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b44ff-31c9-4b99-b358-6a57a9acca79 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.691984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.693035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.693340 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.693583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] PATH_INFO: `/v3/projects/8725d060cfe74d6e8828113cb37832b2/users/e5928c7ff21b43dcb85681f20303b19f/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.711272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8725d060cfe74d6e8828113cb37832b2, user_id=e5928c7ff21b43dcb85681f20303b19f, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.714840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.733714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1186/2361] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:57:55 2022] PUT /identity/v3/projects/8725d060cfe74d6e8828113cb37832b2/users/e5928c7ff21b43dcb85681f20303b19f/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:55.738267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.750889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73af80-fad2-4f35-b48b-b43d6f8c18cc admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.830010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3da99d-9737-4367-9f84-c2bfe002d193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.830881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3da99d-9737-4367-9f84-c2bfe002d193 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.831183 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3da99d-9737-4367-9f84-c2bfe002d193 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.831589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3da99d-9737-4367-9f84-c2bfe002d193 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.832389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3da99d-9737-4367-9f84-c2bfe002d193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.834147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3da99d-9737-4367-9f84-c2bfe002d193 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.847513 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1176/2362] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:55.854430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:55.868913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ca638-2b68-40d4-adfa-30f001db0ea8 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:55.883102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0534bf6-9a01-4490-911c-d166e53ad5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:55.884106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0534bf6-9a01-4490-911c-d166e53ad5b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.884447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0534bf6-9a01-4490-911c-d166e53ad5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.884778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0534bf6-9a01-4490-911c-d166e53ad5b0 admin admin] PATH_INFO: `/v3/projects/8725d060cfe74d6e8828113cb37832b2/users/e5928c7ff21b43dcb85681f20303b19f/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.887643 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0534bf6-9a01-4490-911c-d166e53ad5b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8725d060cfe74d6e8828113cb37832b2, user_id=e5928c7ff21b43dcb85681f20303b19f, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:55.891404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0534bf6-9a01-4490-911c-d166e53ad5b0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:55.904847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1187/2363] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:57:55 2022] PUT /identity/v3/projects/8725d060cfe74d6e8828113cb37832b2/users/e5928c7ff21b43dcb85681f20303b19f/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:57:55.909334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.909798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.910065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.916443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3154d-eaef-4cae-9094-86cd5915d016 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:55.916964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3154d-eaef-4cae-9094-86cd5915d016 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:55.917622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3154d-eaef-4cae-9094-86cd5915d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:55.933993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] MFA Rules not processed for user `aff6489e5a6441509376c0dca551f03c`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:55.935365 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:55.955451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-64c3154d-eaef-4cae-9094-86cd5915d016 None None] MFA Rules not processed for user `e5928c7ff21b43dcb85681f20303b19f`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:55.956733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-64c3154d-eaef-4cae-9094-86cd5915d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:56.017339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1177/2364] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:56.054715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1188/2365] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:57:56.066719 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:56.068331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:56.133697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a80e52-a9db-423a-a041-dd7ad8982f9e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:56.137627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:56.138306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:56.138595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:56.138870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:56.149129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:56.150610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:56.161511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1178/2366] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:57:56.318203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3154d-eaef-4cae-9094-86cd5915d016 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:56.360128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:56.361612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:56.386367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3154d-eaef-4cae-9094-86cd5915d016 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:56.391355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:56.392283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:56.392550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:56.392842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] PATH_INFO: `/v3/users/8999b6906f3e47c697366efafa063288` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:56.394117 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8999b6906f3e47c697366efafa063288)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:56.396096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:56.427143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c962a-18cb-47ff-ae7c-27ce1a76340c service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:56.431638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730c494-8e8c-4b2c-b05e-ec29b5a00db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:56.432946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c494-8e8c-4b2c-b05e-ec29b5a00db6 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:56.433200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c494-8e8c-4b2c-b05e-ec29b5a00db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:56.433642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730c494-8e8c-4b2c-b05e-ec29b5a00db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:56.436760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b730c494-8e8c-4b2c-b05e-ec29b5a00db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:56.453765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730c494-8e8c-4b2c-b05e-ec29b5a00db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:56.455593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730c494-8e8c-4b2c-b05e-ec29b5a00db6 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:56.456597 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8999b6906f3e47c697366efafa063288'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:56.466040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1179/2367] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:57:56.473829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8999b6906f3e47c697366efafa063288'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:56.474959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8999b6906f3e47c697366efafa063288'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:56.475909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8999b6906f3e47c697366efafa063288'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:56.487351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8999b6906f3e47c697366efafa063288'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:56.501564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1189/2368] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:57:56 2022] DELETE /identity/v3/users/8999b6906f3e47c697366efafa063288 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:56.506243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:56.506534 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:56.506812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:56.524964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:56.526278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:56.596481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1180/2369] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:56.603732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:56.605400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:56.674991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d30ba9-0d39-4f06-ac19-a00d2d50f1c9 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:56.679006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:56.679835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:56.680119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:56.680393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:56.690585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:56.692041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:56.703793 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1190/2370] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:57.080833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:57.082445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:57.102154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1675f-52d4-4b9c-b29e-059a6452da87 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:57.126753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:57.127759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:57.129786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:57.130456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] PATH_INFO: `/v3/projects/3b236511c7434bf1bbfc26eb0f2f7818` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:57.136833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b236511c7434bf1bbfc26eb0f2f7818)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:57.136833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:57.209095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b236511c7434bf1bbfc26eb0f2f7818', 'request_id': 'req-cf7e771b-cfa9-4f26-818a-5df52bf2836b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.221852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b236511c7434bf1bbfc26eb0f2f7818', 'request_id': 'req-cf7e771b-cfa9-4f26-818a-5df52bf2836b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.238226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b236511c7434bf1bbfc26eb0f2f7818', 'request_id': 'req-cf7e771b-cfa9-4f26-818a-5df52bf2836b'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.257973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] The token cache is being invalidate because project 3b236511c7434bf1bbfc26eb0f2f7818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:57:57.258333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.259045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.260776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1181/2371] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:57:57 2022] DELETE /identity/v3/projects/3b236511c7434bf1bbfc26eb0f2f7818 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:57.265082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:57.279697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cfef9-27f7-4512-add3-0abf04ead09b service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:57.349678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:57.351059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:57.351452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:57.351814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] PATH_INFO: `/v3/users/e5928c7ff21b43dcb85681f20303b19f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:57.353488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5928c7ff21b43dcb85681f20303b19f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:57.355727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:57.412776 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5928c7ff21b43dcb85681f20303b19f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.430493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5928c7ff21b43dcb85681f20303b19f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.431685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5928c7ff21b43dcb85681f20303b19f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.432420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5928c7ff21b43dcb85681f20303b19f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.442981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a11c6ace-bc8c-455c-b390-ae1af3ec39f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5928c7ff21b43dcb85681f20303b19f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.466552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1191/2372] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:57:57 2022] DELETE /identity/v3/users/e5928c7ff21b43dcb85681f20303b19f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:57.773427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:57.775083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:57.792282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e771b-cfa9-4f26-818a-5df52bf2836b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:57.868396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:57.869415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:57.869730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:57.870027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] PATH_INFO: `/v3/projects/8725d060cfe74d6e8828113cb37832b2` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:57.871268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8725d060cfe74d6e8828113cb37832b2)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:57.873226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:57.940412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8725d060cfe74d6e8828113cb37832b2', 'request_id': 'req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.948058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8725d060cfe74d6e8828113cb37832b2', 'request_id': 'req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.955562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8725d060cfe74d6e8828113cb37832b2', 'request_id': 'req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.972061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] The token cache is being invalidate because project 8725d060cfe74d6e8828113cb37832b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:57:57.972396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.973087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-fc725fd5-339c-40c2-9d8a-ca276a4f7d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:57:57.978639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1182/2373] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:57:57 2022] DELETE /identity/v3/projects/8725d060cfe74d6e8828113cb37832b2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:57.980636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d4f7a-510b-4e37-9c74-c42aa0fbdbef None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:57.980984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d4f7a-510b-4e37-9c74-c42aa0fbdbef None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:57.981299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d4f7a-510b-4e37-9c74-c42aa0fbdbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.000685 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-761d4f7a-510b-4e37-9c74-c42aa0fbdbef None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:58.002362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-761d4f7a-510b-4e37-9c74-c42aa0fbdbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:58.101924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1192/2374] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:57:58.114051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2b2a-6149-47e6-aefe-f4bf5b28ab39 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.114923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2b2a-6149-47e6-aefe-f4bf5b28ab39 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.114923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe2b2a-6149-47e6-aefe-f4bf5b28ab39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.136294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0bfe2b2a-6149-47e6-aefe-f4bf5b28ab39 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:58.137565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0bfe2b2a-6149-47e6-aefe-f4bf5b28ab39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:58.146125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1183/2375] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:58.151614 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d4f7a-510b-4e37-9c74-c42aa0fbdbef None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.167973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d4f7a-510b-4e37-9c74-c42aa0fbdbef None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.173851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.174638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.175784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.175784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.175988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:57:58.176228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.178015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.194735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1193/2376] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:57:58.197427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2b2a-6149-47e6-aefe-f4bf5b28ab39 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.207566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe2b2a-6149-47e6-aefe-f4bf5b28ab39 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.212709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.213609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.214132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.214411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.215212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.217340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.250169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1184/2377] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:57:58 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:57:58.253495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.267824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb222e2-d4fd-4997-b7aa-0309f04644c6 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.338060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.338961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.339241 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.339513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.340291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.342418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.367877 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1194/2378] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:57:58 2022] POST /identity/v3/users => generated 279 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:57:58.370830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.383865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5e64b-6c9f-4561-802a-d4829026e017 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.389748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.390438 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.390733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.391006 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.391758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.393419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.405213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1185/2379] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:58.409594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.420032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9e651-8ddf-46ea-9540-a1b8715c355f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.425355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38438eed-bbec-4381-95e0-fcfce0ca3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.426100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38438eed-bbec-4381-95e0-fcfce0ca3bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.426389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38438eed-bbec-4381-95e0-fcfce0ca3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.426664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38438eed-bbec-4381-95e0-fcfce0ca3bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.427414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38438eed-bbec-4381-95e0-fcfce0ca3bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.429082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38438eed-bbec-4381-95e0-fcfce0ca3bd4 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.445025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1195/2380] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:58.446284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.460497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c14f02-2771-441e-a3d8-7ad419db33db admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.468787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.469907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.470253 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.470667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] PATH_INFO: `/v3/projects/d34fc6785b7c49e3aaeae61617d5135e/users/715ef3016a2442e9bb7301dbd69bd6ec/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.488964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34fc6785b7c49e3aaeae61617d5135e, user_id=715ef3016a2442e9bb7301dbd69bd6ec, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.491608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.507179 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1186/2381] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:57:58 2022] PUT /identity/v3/projects/d34fc6785b7c49e3aaeae61617d5135e/users/715ef3016a2442e9bb7301dbd69bd6ec/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:58.518701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021e069-8afe-4960-9a20-421b135f80a1 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.519419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021e069-8afe-4960-9a20-421b135f80a1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.519711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021e069-8afe-4960-9a20-421b135f80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.560526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-7021e069-8afe-4960-9a20-421b135f80a1 None None] MFA Rules not processed for user `715ef3016a2442e9bb7301dbd69bd6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:58.561928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7021e069-8afe-4960-9a20-421b135f80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:58.641209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1196/2382] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:58.646212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.719401 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db50c62-dd39-46b5-a664-d47ccaf810fc admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.724626 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.725339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.725620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.725907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.726776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.728724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.759620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1187/2383] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:57:58 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:57:58.763892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021e069-8afe-4960-9a20-421b135f80a1 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.781342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021e069-8afe-4960-9a20-421b135f80a1 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.873905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.875893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.876833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.876833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.878173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.881291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.912362 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1197/2384] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:57:58 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:57:58.917843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:58.945282 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9e1bb-0362-4069-b074-08c04df10720 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:58.954924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:58.955632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:58.955902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:58.956346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:58.957182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:58.958862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:58.980625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1188/2385] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:58.988397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:59.005234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba8071-8be1-461a-996a-31a65bb38e29 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:59.015686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:59.020088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:59.020883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:59.021434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] PATH_INFO: `/v3/projects/b3247025e66848a8b4ecb90350d70ef0/users/b10ada2a03de4790afef637742200bf5/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:59.050274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3247025e66848a8b4ecb90350d70ef0, user_id=b10ada2a03de4790afef637742200bf5, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:59.055628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:59.081642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1198/2386] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:57:58 2022] PUT /identity/v3/projects/b3247025e66848a8b4ecb90350d70ef0/users/b10ada2a03de4790afef637742200bf5/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:57:59.085959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:59.104808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cd8ca-c4d1-4cf5-bebd-5bf726529ad8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:59.222224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ebc6-f97a-4a05-b3d5-cf318396a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:59.224954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ebc6-f97a-4a05-b3d5-cf318396a0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:59.225470 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ebc6-f97a-4a05-b3d5-cf318396a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:59.226294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ebc6-f97a-4a05-b3d5-cf318396a0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:59.228039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40ebc6-f97a-4a05-b3d5-cf318396a0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:59.231592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40ebc6-f97a-4a05-b3d5-cf318396a0c0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:59.253485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1189/2387] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:57:59.257401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:59.269999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2b86-075b-4be3-ac8a-4f395241fbd7 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:59.276342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:59.277488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:59.277849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:59.278124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] PATH_INFO: `/v3/projects/b3247025e66848a8b4ecb90350d70ef0/users/b10ada2a03de4790afef637742200bf5/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:59.280559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3247025e66848a8b4ecb90350d70ef0, user_id=b10ada2a03de4790afef637742200bf5, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:59.283477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:59.312860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1199/2388] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:57:59 2022] PUT /identity/v3/projects/b3247025e66848a8b4ecb90350d70ef0/users/b10ada2a03de4790afef637742200bf5/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:57:59.313904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa4925-4978-4069-8c70-2758dc0f3def None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:59.313904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa4925-4978-4069-8c70-2758dc0f3def None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:59.313904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa4925-4978-4069-8c70-2758dc0f3def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:59.347102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-3afa4925-4978-4069-8c70-2758dc0f3def None None] MFA Rules not processed for user `b10ada2a03de4790afef637742200bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:57:59.348516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3afa4925-4978-4069-8c70-2758dc0f3def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:59.456620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1190/2389] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:57:59.470611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:59.472392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:57:59.561960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bb09f-374a-4b62-8671-6737087cf567 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:59.566047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:59.566984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:59.567386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:59.567749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:59.580956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:59.582411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:59.594342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1200/2390] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:59 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:57:59.686628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa4925-4978-4069-8c70-2758dc0f3def None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:57:59.700595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa4925-4978-4069-8c70-2758dc0f3def None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:57:59.704300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:57:59.705066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:57:59.705343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:57:59.705608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:57:59.805593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:57:59.805593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:57:59.815819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1191/2391] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:57:59 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:06.140416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:06.142158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:06.230557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ce5d-4a9b-4b92-9f83-0227ff71674b service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:06.236622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca12cfb-e427-4887-8649-b7feb118ad4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:06.237182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca12cfb-e427-4887-8649-b7feb118ad4f service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:06.237463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca12cfb-e427-4887-8649-b7feb118ad4f service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:06.237799 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca12cfb-e427-4887-8649-b7feb118ad4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:06.239641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7ca12cfb-e427-4887-8649-b7feb118ad4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:06.252731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca12cfb-e427-4887-8649-b7feb118ad4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:06.254407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca12cfb-e427-4887-8649-b7feb118ad4f service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:06.267222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1201/2392] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:09.454981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:09.457191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:09.478465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cc725-d114-4b34-8404-1f5c88faffa3 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:09.486225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f42b8-5760-4647-9fc2-9a3f4107847f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:09.486944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f42b8-5760-4647-9fc2-9a3f4107847f service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:09.487223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f42b8-5760-4647-9fc2-9a3f4107847f service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:09.487499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f42b8-5760-4647-9fc2-9a3f4107847f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:09.489364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d66f42b8-5760-4647-9fc2-9a3f4107847f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:09.502374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f42b8-5760-4647-9fc2-9a3f4107847f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:09.503993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f42b8-5760-4647-9fc2-9a3f4107847f service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:09.518286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1192/2393] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:09.591522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:09.592455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:09.593037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:09.614142 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] MFA Rules not processed for user `3075413bbcba4104bb5fbb907ab2558a`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:09.615672 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:09.704878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1202/2394] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:10.226107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a8b82-c872-4aa8-9067-b0e6bfe082a4 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:10.226435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a8b82-c872-4aa8-9067-b0e6bfe082a4 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:10.227307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a8b82-c872-4aa8-9067-b0e6bfe082a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:10.254721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-726a8b82-c872-4aa8-9067-b0e6bfe082a4 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:10.256140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-726a8b82-c872-4aa8-9067-b0e6bfe082a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:10.389848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1193/2395] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:10.478971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:10.485714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:10.621103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b8290-b7b2-4da4-b32a-fd13f4eea84c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:10.636770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d610bb3-35da-46a5-b096-08c4a1ffab47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:10.640780 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610bb3-35da-46a5-b096-08c4a1ffab47 service swift] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:10.642777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610bb3-35da-46a5-b096-08c4a1ffab47 service swift] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:10.643173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d610bb3-35da-46a5-b096-08c4a1ffab47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:10.655868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610bb3-35da-46a5-b096-08c4a1ffab47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:10.657809 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d610bb3-35da-46a5-b096-08c4a1ffab47 service swift] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:10.670400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1203/2396] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:58:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 14:58:16.003900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a8b82-c872-4aa8-9067-b0e6bfe082a4 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:16.094742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a8b82-c872-4aa8-9067-b0e6bfe082a4 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:16.099606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:16.100606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:16.100911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:16.101193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] PATH_INFO: `/v3/users/715ef3016a2442e9bb7301dbd69bd6ec` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:16.102737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] RBAC: Authorizing `identity:delete_user(user_id=715ef3016a2442e9bb7301dbd69bd6ec)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:16.105776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:16.174853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715ef3016a2442e9bb7301dbd69bd6ec'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:16.175599 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715ef3016a2442e9bb7301dbd69bd6ec'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:16.176272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715ef3016a2442e9bb7301dbd69bd6ec'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:16.187080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715ef3016a2442e9bb7301dbd69bd6ec'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:16.205384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715ef3016a2442e9bb7301dbd69bd6ec'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:16.224366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1194/2397] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:16 2022] DELETE /identity/v3/users/715ef3016a2442e9bb7301dbd69bd6ec => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:16.229358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:16.230596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:16.231195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:16.256097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:16.258393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:16.393934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1204/2398] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:16.404190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:16.407941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:16.539286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f66382-dfce-4831-a855-445a8ade756f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:16.560318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:16.562877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:16.563497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:16.565337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:16.586166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:16.589102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:16.611127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1195/2399] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:16.963186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:16.965178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:16.979509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b058d-6906-4f60-a75a-1851d9d1a91b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:16.996555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:16.997447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:16.997713 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:16.997984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] PATH_INFO: `/v3/projects/d34fc6785b7c49e3aaeae61617d5135e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:16.999341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34fc6785b7c49e3aaeae61617d5135e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:17.001350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:17.064834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34fc6785b7c49e3aaeae61617d5135e', 'request_id': 'req-5dd0bedd-5474-40c6-b09a-11a972d045f0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.074574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34fc6785b7c49e3aaeae61617d5135e', 'request_id': 'req-5dd0bedd-5474-40c6-b09a-11a972d045f0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.085312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34fc6785b7c49e3aaeae61617d5135e', 'request_id': 'req-5dd0bedd-5474-40c6-b09a-11a972d045f0'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.106021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] The token cache is being invalidate because project d34fc6785b7c49e3aaeae61617d5135e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:17.106374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.107155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.109618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1205/2400] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:16 2022] DELETE /identity/v3/projects/d34fc6785b7c49e3aaeae61617d5135e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:17.114329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:17.131390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76ae8-11b0-4a69-925a-14b61574806e service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:17.225980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:17.228035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:17.228520 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:17.228803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] PATH_INFO: `/v3/users/b10ada2a03de4790afef637742200bf5` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:17.230379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b10ada2a03de4790afef637742200bf5)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:17.232618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:17.294463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b10ada2a03de4790afef637742200bf5'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.295167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b10ada2a03de4790afef637742200bf5'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.295777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b10ada2a03de4790afef637742200bf5'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.305050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b10ada2a03de4790afef637742200bf5'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.324727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-efdbc146-9ecc-4583-97f4-3bd36285e7cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b10ada2a03de4790afef637742200bf5'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.342783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1196/2401] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:17 2022] DELETE /identity/v3/users/b10ada2a03de4790afef637742200bf5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:17.734044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:17.736593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:17.770394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0bedd-5474-40c6-b09a-11a972d045f0 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:17.850094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:17.851516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:17.851516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:17.851734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] PATH_INFO: `/v3/projects/b3247025e66848a8b4ecb90350d70ef0` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:17.853134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3247025e66848a8b4ecb90350d70ef0)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:17.855205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:17.916373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3247025e66848a8b4ecb90350d70ef0', 'request_id': 'req-fb3fb712-6aac-43c8-8c68-339298759b64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.925629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3247025e66848a8b4ecb90350d70ef0', 'request_id': 'req-fb3fb712-6aac-43c8-8c68-339298759b64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.934132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3247025e66848a8b4ecb90350d70ef0', 'request_id': 'req-fb3fb712-6aac-43c8-8c68-339298759b64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.946258 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] The token cache is being invalidate because project b3247025e66848a8b4ecb90350d70ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:17.946610 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.947279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb3fb712-6aac-43c8-8c68-339298759b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:17.950021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1206/2402] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:17 2022] DELETE /identity/v3/projects/b3247025e66848a8b4ecb90350d70ef0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:17.955629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b439-8eb7-4f87-af50-7c706dd46007 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:17.956597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b439-8eb7-4f87-af50-7c706dd46007 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:17.956908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b439-8eb7-4f87-af50-7c706dd46007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:17.976114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-1e24b439-8eb7-4f87-af50-7c706dd46007 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:17.977569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-1e24b439-8eb7-4f87-af50-7c706dd46007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:18.049893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1197/2403] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:58:18.058334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0459-1942-4a9f-8b3d-ba6cdf0213df None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.058629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0459-1942-4a9f-8b3d-ba6cdf0213df None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.058910 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d0459-1942-4a9f-8b3d-ba6cdf0213df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.079302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d56d0459-1942-4a9f-8b3d-ba6cdf0213df None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:18.081020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d56d0459-1942-4a9f-8b3d-ba6cdf0213df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:18.089134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1207/2404] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:18.093673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24b439-8eb7-4f87-af50-7c706dd46007 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.105573 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24b439-8eb7-4f87-af50-7c706dd46007 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.110843 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.111593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.111947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.112430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.113287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:58:18.113663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.115357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:18.132434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1198/2405] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:58:18.133716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0459-1942-4a9f-8b3d-ba6cdf0213df None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.143798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d0459-1942-4a9f-8b3d-ba6cdf0213df None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.148673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.149442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.149813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.150083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.150783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.152828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:18.189186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1208/2406] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:58:18 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:18.190054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.201488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d11917c-8c52-48c0-a942-a3ab46faec20 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.281415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.282202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.282504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.282804 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.283810 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.286071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:18.312139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1199/2407] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:58:18 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:58:18.316023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.336848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7c64f-4f4b-4b72-99b3-2ac3f9124903 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.344359 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.345091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.345369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.345638 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.346397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.347988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:18.362463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1209/2408] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:18.365295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.378242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc76020-419e-4a04-bac2-9f7490d3ecea admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.387301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6041e93-ce63-4dae-9a31-da5eb806a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.388019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6041e93-ce63-4dae-9a31-da5eb806a4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.388312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6041e93-ce63-4dae-9a31-da5eb806a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.388574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6041e93-ce63-4dae-9a31-da5eb806a4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.389511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6041e93-ce63-4dae-9a31-da5eb806a4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.391266 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6041e93-ce63-4dae-9a31-da5eb806a4e0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:18.406918 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1200/2409] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:18.410813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.423520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf016ecd-a4fe-4e6f-aef9-30e5d7bf955d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.429134 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.430466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.430839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.431202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] PATH_INFO: `/v3/projects/c0ba1b765e9c4ffbbedc5e8ad55a307c/users/ac8247d4178947e6a7cf654e28a88cad/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.477583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ba1b765e9c4ffbbedc5e8ad55a307c, user_id=ac8247d4178947e6a7cf654e28a88cad, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.480348 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:18.518787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1210/2410] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:18 2022] PUT /identity/v3/projects/c0ba1b765e9c4ffbbedc5e8ad55a307c/users/ac8247d4178947e6a7cf654e28a88cad/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:58:18.522085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9847-f681-4790-9f6f-c52fd0d14a2d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.522666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9847-f681-4790-9f6f-c52fd0d14a2d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.523020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9847-f681-4790-9f6f-c52fd0d14a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.562135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d7be9847-f681-4790-9f6f-c52fd0d14a2d None None] MFA Rules not processed for user `ac8247d4178947e6a7cf654e28a88cad`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:18.563449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d7be9847-f681-4790-9f6f-c52fd0d14a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:18.660420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1201/2411] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:18.665513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.805717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d505dd-bd6a-4f4f-893b-04159ae38d9c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.811615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.812327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.812607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.813071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.814066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.816061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:18.866767 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1211/2412] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:58:18 2022] POST /identity/v3/projects => generated 389 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:18.872043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be9847-f681-4790-9f6f-c52fd0d14a2d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:18.888977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be9847-f681-4790-9f6f-c52fd0d14a2d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:18.984702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:18.985568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:18.985849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:18.986164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:18.986894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:18.991210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:19.024001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1202/2413] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:58:18 2022] POST /identity/v3/users => generated 301 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:19.029167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:19.042048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e1851-9d1e-464a-b778-cece6b5bb556 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:19.047954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:19.049096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.049592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.050918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.051923 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:19.053842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:19.066512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1212/2414] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:19.070789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:19.082391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf405296-792b-4710-b9e0-c48fe372d5f3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:19.087845 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:19.088762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.089071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.089419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] PATH_INFO: `/v3/projects/2546391c448f4e7ca451e12689174d82/users/b4f88bdde05b46ec9204bf9e3c44ef7a/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.107635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2546391c448f4e7ca451e12689174d82, user_id=b4f88bdde05b46ec9204bf9e3c44ef7a, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:19.111526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:19.130535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1203/2415] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:19 2022] PUT /identity/v3/projects/2546391c448f4e7ca451e12689174d82/users/b4f88bdde05b46ec9204bf9e3c44ef7a/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:58:19.137833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:19.150409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d53c9-7944-40f3-b8af-0446c776f52d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:19.224601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032b7bb-46ae-48d2-a259-2d19719dc191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:19.225422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032b7bb-46ae-48d2-a259-2d19719dc191 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.225783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032b7bb-46ae-48d2-a259-2d19719dc191 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.226062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032b7bb-46ae-48d2-a259-2d19719dc191 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.226835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032b7bb-46ae-48d2-a259-2d19719dc191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:19.228517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032b7bb-46ae-48d2-a259-2d19719dc191 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:19.245814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1213/2416] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:19.247201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:19.262877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3565e7-53cb-432a-bd85-a51277c1511b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:19.267860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:19.268852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.269110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.269380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] PATH_INFO: `/v3/projects/2546391c448f4e7ca451e12689174d82/users/b4f88bdde05b46ec9204bf9e3c44ef7a/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.271387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2546391c448f4e7ca451e12689174d82, user_id=b4f88bdde05b46ec9204bf9e3c44ef7a, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:19.274098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:19.287070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1204/2417] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:19 2022] PUT /identity/v3/projects/2546391c448f4e7ca451e12689174d82/users/b4f88bdde05b46ec9204bf9e3c44ef7a/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:19.301260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8881d-d07e-41c5-8c84-dacc2ce75c22 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.301552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8881d-d07e-41c5-8c84-dacc2ce75c22 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.301825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8881d-d07e-41c5-8c84-dacc2ce75c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.339052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-78f8881d-d07e-41c5-8c84-dacc2ce75c22 None None] MFA Rules not processed for user `b4f88bdde05b46ec9204bf9e3c44ef7a`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:19.340560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-78f8881d-d07e-41c5-8c84-dacc2ce75c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:19.420761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1214/2418] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:19.432584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:19.434785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:19.525638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2555c0-d316-4bca-8205-eae8e9fccec3 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:19.532126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fed542-5cd4-4aa3-94f8-5c146958785a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:19.532814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fed542-5cd4-4aa3-94f8-5c146958785a service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.533099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fed542-5cd4-4aa3-94f8-5c146958785a service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.533372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fed542-5cd4-4aa3-94f8-5c146958785a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.543421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fed542-5cd4-4aa3-94f8-5c146958785a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:19.544917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fed542-5cd4-4aa3-94f8-5c146958785a service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:19.559133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1205/2419] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:19.738669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8881d-d07e-41c5-8c84-dacc2ce75c22 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:19.818443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8881d-d07e-41c5-8c84-dacc2ce75c22 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:19.824265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:19.825309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.825540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.825880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] PATH_INFO: `/v3/users/ac8247d4178947e6a7cf654e28a88cad` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.827313 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8247d4178947e6a7cf654e28a88cad)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:19.829335 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:19.891438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8247d4178947e6a7cf654e28a88cad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:19.909176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8247d4178947e6a7cf654e28a88cad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:19.910044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8247d4178947e6a7cf654e28a88cad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:19.911233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8247d4178947e6a7cf654e28a88cad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:19.925065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8247d4178947e6a7cf654e28a88cad'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:19.958189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1215/2420] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:19 2022] DELETE /identity/v3/users/ac8247d4178947e6a7cf654e28a88cad => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:19.963043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:19.963337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:19.963622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:19.989521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:19.989521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:20.068241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1206/2421] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:58:20.078301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:20.079908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:20.170585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58489c73-700a-442f-a550-c03c100d285e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:20.174897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:20.175633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:20.175915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:20.176093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:20.186972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:20.188552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:20.204172 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1216/2422] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:20.592864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:20.594039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:20.607358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40ece1-29ba-438a-9e6e-dd69e2a8264c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:20.622341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:20.623576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:20.624639 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:20.625155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] PATH_INFO: `/v3/projects/c0ba1b765e9c4ffbbedc5e8ad55a307c` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:20.627107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ba1b765e9c4ffbbedc5e8ad55a307c)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:20.630043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:20.731456 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ba1b765e9c4ffbbedc5e8ad55a307c', 'request_id': 'req-7e978856-0a0d-46c3-b5e1-9780500671a4'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:20.739500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ba1b765e9c4ffbbedc5e8ad55a307c', 'request_id': 'req-7e978856-0a0d-46c3-b5e1-9780500671a4'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:20.746668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ba1b765e9c4ffbbedc5e8ad55a307c', 'request_id': 'req-7e978856-0a0d-46c3-b5e1-9780500671a4'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:20.775045 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] The token cache is being invalidate because project c0ba1b765e9c4ffbbedc5e8ad55a307c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:20.775390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:20.776262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:20.786167 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1207/2423] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:20 2022] DELETE /identity/v3/projects/c0ba1b765e9c4ffbbedc5e8ad55a307c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:20.788261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:20.809271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae7cf8-817c-43d9-9687-2463e1bb7c25 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:20.910723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:20.911644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:20.911925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:20.912195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] PATH_INFO: `/v3/users/b4f88bdde05b46ec9204bf9e3c44ef7a` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:20.913690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f88bdde05b46ec9204bf9e3c44ef7a)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:20.915674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:20.984127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f88bdde05b46ec9204bf9e3c44ef7a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:20.998460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f88bdde05b46ec9204bf9e3c44ef7a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:20.999619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f88bdde05b46ec9204bf9e3c44ef7a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.000261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f88bdde05b46ec9204bf9e3c44ef7a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.011490 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-54de968e-fa8f-4fb5-9dd9-6056d1bf2b8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f88bdde05b46ec9204bf9e3c44ef7a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.032763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1217/2424] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:20 2022] DELETE /identity/v3/users/b4f88bdde05b46ec9204bf9e3c44ef7a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:21.448881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:21.450344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:21.463406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978856-0a0d-46c3-b5e1-9780500671a4 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:21.534944 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:21.535901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:21.536494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:21.536775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] PATH_INFO: `/v3/projects/2546391c448f4e7ca451e12689174d82` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:21.538310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2546391c448f4e7ca451e12689174d82)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:21.540235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:21.596637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2546391c448f4e7ca451e12689174d82', 'request_id': 'req-34b51af5-a619-4be9-be84-4329ef5e84b7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.603808 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2546391c448f4e7ca451e12689174d82', 'request_id': 'req-34b51af5-a619-4be9-be84-4329ef5e84b7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.613078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2546391c448f4e7ca451e12689174d82', 'request_id': 'req-34b51af5-a619-4be9-be84-4329ef5e84b7'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.630596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] The token cache is being invalidate because project 2546391c448f4e7ca451e12689174d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:21.630886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.631550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-34b51af5-a619-4be9-be84-4329ef5e84b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:21.633576 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1208/2425] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:21 2022] DELETE /identity/v3/projects/2546391c448f4e7ca451e12689174d82 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:21.638933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2ff83-4034-47bd-9b22-16cd364e9884 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:21.639214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2ff83-4034-47bd-9b22-16cd364e9884 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:21.639509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2ff83-4034-47bd-9b22-16cd364e9884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:21.658800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-22e2ff83-4034-47bd-9b22-16cd364e9884 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:21.660422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-22e2ff83-4034-47bd-9b22-16cd364e9884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:21.720711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1218/2426] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:21.729048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149b2c-2328-4161-b511-84f8511dd88c None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:21.729336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149b2c-2328-4161-b511-84f8511dd88c None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:21.729614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149b2c-2328-4161-b511-84f8511dd88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:21.751699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6c149b2c-2328-4161-b511-84f8511dd88c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:21.753004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6c149b2c-2328-4161-b511-84f8511dd88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:21.762632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1209/2427] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:21.767399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2ff83-4034-47bd-9b22-16cd364e9884 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:21.778491 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2ff83-4034-47bd-9b22-16cd364e9884 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:21.783889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:21.784609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:21.784932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:21.785213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:21.785964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:58:21.786268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:21.787903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:21.800916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1219/2428] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:58:21.806160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c149b2c-2328-4161-b511-84f8511dd88c None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:21.815972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c149b2c-2328-4161-b511-84f8511dd88c None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:21.823201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:21.824034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:21.824328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:21.824605 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:21.825388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:21.827330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:21.863216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1210/2429] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:58:21 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:21.864406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:21.877068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028811e2-3437-48e5-9ab9-bd27eafbfeec admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:21.941803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:21.942568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:21.944492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:21.944492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:21.944492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:21.945871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:21.987074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1220/2430] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:58:21 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:21.988735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.002799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52099de1-38b1-4446-8434-5325729c4cb5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.010840 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.011685 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.012049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.012333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.013162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.015275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.029214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1211/2431] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:22.033500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.046727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316e48f-8a78-4d0b-ac63-67b2c4e0b06d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.051764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b822f-7d5f-4602-b569-404fd68b05ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.052570 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b822f-7d5f-4602-b569-404fd68b05ac admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.052947 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b822f-7d5f-4602-b569-404fd68b05ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.053312 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b822f-7d5f-4602-b569-404fd68b05ac admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.054242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b822f-7d5f-4602-b569-404fd68b05ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.056276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b822f-7d5f-4602-b569-404fd68b05ac admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.067995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1221/2432] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:58:22.072823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.084312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaa71e-0505-4f1f-ba36-9188b1172973 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.090341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.091321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.091625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.091900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] PATH_INFO: `/v3/projects/02231ed02abc4704b9246e810a1849de/users/b6b7ece29fbe40f981b66199d5034d63/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.111812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02231ed02abc4704b9246e810a1849de, user_id=b6b7ece29fbe40f981b66199d5034d63, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.115099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.131675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1212/2433] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:22 2022] PUT /identity/v3/projects/02231ed02abc4704b9246e810a1849de/users/b6b7ece29fbe40f981b66199d5034d63/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:22.143438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841ea2-01c6-4e7a-bc97-4f189e4dc3ba None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.143712 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841ea2-01c6-4e7a-bc97-4f189e4dc3ba None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.143987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5841ea2-01c6-4e7a-bc97-4f189e4dc3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.172704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f5841ea2-01c6-4e7a-bc97-4f189e4dc3ba None None] MFA Rules not processed for user `b6b7ece29fbe40f981b66199d5034d63`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:22.173778 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f5841ea2-01c6-4e7a-bc97-4f189e4dc3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:22.244237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1222/2434] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:22.248846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.318903 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e9c1-77bc-465f-bf14-44cb593d0914 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.324827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.326010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.326010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.326010 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.326757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.328775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.368332 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1213/2435] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:58:22 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:22.369805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5841ea2-01c6-4e7a-bc97-4f189e4dc3ba None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.384012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5841ea2-01c6-4e7a-bc97-4f189e4dc3ba None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.458137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.459058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.459440 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.459876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.460682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.462730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.488240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1223/2436] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:58:22 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:22.489751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.500154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3d130-a5be-47f7-ae2f-d3282ffb5439 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.504988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.505735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.506123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.506491 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.507731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.509389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.525379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1214/2437] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:22.527265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.538596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605497da-013c-47c7-acb4-82b541f979f0 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.543432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.544350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.544633 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.544964 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] PATH_INFO: `/v3/projects/643aca7e7f194749b2ee55817a35032a/users/b94cfd04e71e4f69ab4ca766c360ea29/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.560455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643aca7e7f194749b2ee55817a35032a, user_id=b94cfd04e71e4f69ab4ca766c360ea29, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.563192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.577743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1224/2438] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:22 2022] PUT /identity/v3/projects/643aca7e7f194749b2ee55817a35032a/users/b94cfd04e71e4f69ab4ca766c360ea29/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:22.582060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.592767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2fff-1adf-48d9-a126-ae1095c3e3f4 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.654373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913bb30-de07-4b63-a20b-45b5edd12376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.655120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913bb30-de07-4b63-a20b-45b5edd12376 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.655695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913bb30-de07-4b63-a20b-45b5edd12376 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.655974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913bb30-de07-4b63-a20b-45b5edd12376 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.656707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913bb30-de07-4b63-a20b-45b5edd12376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.658386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913bb30-de07-4b63-a20b-45b5edd12376 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.669290 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1215/2439] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:58:22.674747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.686049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0f667-c7ef-4190-b2f8-95a5a3914cb1 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.691593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.692476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.692761 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.693044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] PATH_INFO: `/v3/projects/643aca7e7f194749b2ee55817a35032a/users/b94cfd04e71e4f69ab4ca766c360ea29/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.695112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=643aca7e7f194749b2ee55817a35032a, user_id=b94cfd04e71e4f69ab4ca766c360ea29, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.697816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.708647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1225/2440] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:22 2022] PUT /identity/v3/projects/643aca7e7f194749b2ee55817a35032a/users/b94cfd04e71e4f69ab4ca766c360ea29/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:22.720820 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1b5c-22b1-4aa5-bbe2-e0c5efe12cda None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.721235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1b5c-22b1-4aa5-bbe2-e0c5efe12cda None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.721622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1b5c-22b1-4aa5-bbe2-e0c5efe12cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.767371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-18ee1b5c-22b1-4aa5-bbe2-e0c5efe12cda None None] MFA Rules not processed for user `b94cfd04e71e4f69ab4ca766c360ea29`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:22.768718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-18ee1b5c-22b1-4aa5-bbe2-e0c5efe12cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:22.846879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1216/2441] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:22.859582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:22.861786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:22.923044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36efb5-9e1a-489a-a07a-afd641a99d0f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:22.927609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912e686-81c7-4ad8-b627-ba261149b8ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:22.928251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912e686-81c7-4ad8-b627-ba261149b8ed service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:22.928530 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912e686-81c7-4ad8-b627-ba261149b8ed service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:22.928827 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912e686-81c7-4ad8-b627-ba261149b8ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:22.938690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912e686-81c7-4ad8-b627-ba261149b8ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:22.940127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912e686-81c7-4ad8-b627-ba261149b8ed service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:22.949652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1226/2442] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:23.046287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee1b5c-22b1-4aa5-bbe2-e0c5efe12cda None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:23.057236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee1b5c-22b1-4aa5-bbe2-e0c5efe12cda None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:23.062397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb8f9b-62c3-4e67-809f-0b73e37afea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:23.063041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8f9b-62c3-4e67-809f-0b73e37afea2 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:23.063318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8f9b-62c3-4e67-809f-0b73e37afea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:23.063586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8f9b-62c3-4e67-809f-0b73e37afea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:23.129912 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb8f9b-62c3-4e67-809f-0b73e37afea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:23.131344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb8f9b-62c3-4e67-809f-0b73e37afea2 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:23.141637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1217/2443] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:25.399222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d98e0b-b384-40a9-be8b-c9089474d644 None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.399803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d98e0b-b384-40a9-be8b-c9089474d644 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.399803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d98e0b-b384-40a9-be8b-c9089474d644 None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.401397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1227/2444] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 04 14:58:25.407721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a964-9ae6-4e72-b4f5-0382fd2c4f43 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.408227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a964-9ae6-4e72-b4f5-0382fd2c4f43 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.408405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551a964-9ae6-4e72-b4f5-0382fd2c4f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.427118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7551a964-9ae6-4e72-b4f5-0382fd2c4f43 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:25.428448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7551a964-9ae6-4e72-b4f5-0382fd2c4f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:25.500195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1218/2445] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:25.513684 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d98e0b-b384-40a9-be8b-c9089474d644 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:25.515828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e1d98e0b-b384-40a9-be8b-c9089474d644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:25.591266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d98e0b-b384-40a9-be8b-c9089474d644 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:25.594975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ff723-c8c2-4586-a568-675d05d27b29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:25.595642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ff723-c8c2-4586-a568-675d05d27b29 service glance] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.595918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ff723-c8c2-4586-a568-675d05d27b29 service glance] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.596188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ff723-c8c2-4586-a568-675d05d27b29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.607231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ff723-c8c2-4586-a568-675d05d27b29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:25.609537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ff723-c8c2-4586-a568-675d05d27b29 service glance] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:25.625143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1228/2446] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:25.708454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6070c-313d-48ca-aaa1-c1f93fcf7aee None None] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.708814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6070c-313d-48ca-aaa1-c1f93fcf7aee None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.709099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6070c-313d-48ca-aaa1-c1f93fcf7aee None None] PATH_INFO: `/` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.710517 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1219/2447] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:58:25.716781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dfbc5-7650-46a4-b176-27026a8c3cea None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.717323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dfbc5-7650-46a4-b176-27026a8c3cea None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.717654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dfbc5-7650-46a4-b176-27026a8c3cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.746299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0a4dfbc5-7650-46a4-b176-27026a8c3cea None None] MFA Rules not processed for user `0d678d5ee7d74896bbc1316b32ec81ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:25.747510 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0a4dfbc5-7650-46a4-b176-27026a8c3cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:25.812646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1229/2448] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:25.826486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6070c-313d-48ca-aaa1-c1f93fcf7aee None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:25.828296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-92e6070c-313d-48ca-aaa1-c1f93fcf7aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:25.842589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6070c-313d-48ca-aaa1-c1f93fcf7aee None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:25.846313 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61674b9-8fa8-4927-a136-d1c1716b767f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:25.846985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61674b9-8fa8-4927-a136-d1c1716b767f service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.847268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61674b9-8fa8-4927-a136-d1c1716b767f service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.847542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61674b9-8fa8-4927-a136-d1c1716b767f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.855706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61674b9-8fa8-4927-a136-d1c1716b767f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:25.857247 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61674b9-8fa8-4927-a136-d1c1716b767f service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:25.866285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1220/2449] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:25.970065 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb6b8-bc57-4828-b2f6-953c7b769e2c None None] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.970353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb6b8-bc57-4828-b2f6-953c7b769e2c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.970679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5cb6b8-bc57-4828-b2f6-953c7b769e2c None None] PATH_INFO: `/` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.972231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1230/2450] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 4 14:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 04 14:58:25.978150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d5cd8-36a6-416b-9fe8-b46fbc958b10 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:25.978437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d5cd8-36a6-416b-9fe8-b46fbc958b10 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:25.978718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d5cd8-36a6-416b-9fe8-b46fbc958b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:25.999120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-cc8d5cd8-36a6-416b-9fe8-b46fbc958b10 None None] MFA Rules not processed for user `753f2914e5bc4e298b99529e230d9ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:26.000322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cc8d5cd8-36a6-416b-9fe8-b46fbc958b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:26.011463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1221/2451] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 4 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:26.023419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cb6b8-bc57-4828-b2f6-953c7b769e2c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:26.025415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8c5cb6b8-bc57-4828-b2f6-953c7b769e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:26.041382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5cb6b8-bc57-4828-b2f6-953c7b769e2c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:26.045801 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47a313-209c-4aa1-9c8e-4e00eedfd35a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:26.046467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a313-209c-4aa1-9c8e-4e00eedfd35a service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:26.046748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a313-209c-4aa1-9c8e-4e00eedfd35a service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:26.047405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a313-209c-4aa1-9c8e-4e00eedfd35a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:26.069262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47a313-209c-4aa1-9c8e-4e00eedfd35a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:26.070917 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47a313-209c-4aa1-9c8e-4e00eedfd35a service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:26.086693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1231/2452] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:28.169900 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d5cd8-36a6-416b-9fe8-b46fbc958b10 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:28.247832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d5cd8-36a6-416b-9fe8-b46fbc958b10 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:28.253740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:28.254669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:28.254948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:28.255225 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] PATH_INFO: `/v3/users/b6b7ece29fbe40f981b66199d5034d63` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:28.256563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b7ece29fbe40f981b66199d5034d63)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:28.258916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:28.330572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b7ece29fbe40f981b66199d5034d63'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:28.331919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b7ece29fbe40f981b66199d5034d63'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:28.333170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b7ece29fbe40f981b66199d5034d63'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:28.348747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b7ece29fbe40f981b66199d5034d63'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:28.366620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b7ece29fbe40f981b66199d5034d63'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:28.385343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1222/2453] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:28 2022] DELETE /identity/v3/users/b6b7ece29fbe40f981b66199d5034d63 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:28.391983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de629f36-fc2b-4ba7-9263-bc36b1e94e9a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:28.392907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de629f36-fc2b-4ba7-9263-bc36b1e94e9a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:28.392907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de629f36-fc2b-4ba7-9263-bc36b1e94e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:28.413623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-de629f36-fc2b-4ba7-9263-bc36b1e94e9a None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:28.415089 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-de629f36-fc2b-4ba7-9263-bc36b1e94e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:28.522738 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1232/2454] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:28.548147 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:28.550972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:28.632542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552908c-c5fb-44ac-b2d6-b101a547473c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:28.636765 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:28.637484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:28.637767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:28.638040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:28.651911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:28.653931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:28.673428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1223/2455] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:29.000448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:29.000847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:29.001200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:29.029702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:29.032329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:29.151584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1233/2456] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 14:58:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:58:29.159454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:29.162657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:29.288023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb43ab4-5248-4021-b94c-447e19997ecc service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:29.297184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:29.300020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:29.300510 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:29.300993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:29.326742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:29.330398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:29.356161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1224/2457] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:58:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 04 14:58:29.509324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:29.511354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:29.569114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b6bf5-88fc-4864-8f0d-4ebeac7c91fc None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:29.583320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:29.585165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:29.585958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:29.586719 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] PATH_INFO: `/v3/projects/02231ed02abc4704b9246e810a1849de` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:29.588905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=02231ed02abc4704b9246e810a1849de)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:29.591526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:29.681291 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02231ed02abc4704b9246e810a1849de', 'request_id': 'req-51309fe2-92fd-4c91-accd-81c873ceadbd'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.695989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02231ed02abc4704b9246e810a1849de', 'request_id': 'req-51309fe2-92fd-4c91-accd-81c873ceadbd'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.710136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02231ed02abc4704b9246e810a1849de', 'request_id': 'req-51309fe2-92fd-4c91-accd-81c873ceadbd'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.730506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] The token cache is being invalidate because project 02231ed02abc4704b9246e810a1849de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:29.731424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.732784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.735187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1234/2458] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:29 2022] DELETE /identity/v3/projects/02231ed02abc4704b9246e810a1849de => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:29.739701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:29.757490 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3116714-2f8d-40c6-a899-b1a7bcc4d568 service swift] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:29.894731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:29.895842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:29.896120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:29.896397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] PATH_INFO: `/v3/users/b94cfd04e71e4f69ab4ca766c360ea29` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:29.897829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94cfd04e71e4f69ab4ca766c360ea29)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:29.900273 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:29.977148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94cfd04e71e4f69ab4ca766c360ea29'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.978250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94cfd04e71e4f69ab4ca766c360ea29'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.979531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94cfd04e71e4f69ab4ca766c360ea29'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:29.990318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94cfd04e71e4f69ab4ca766c360ea29'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:30.014121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-83f09964-f60b-471a-840e-cf7aad3c0fff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94cfd04e71e4f69ab4ca766c360ea29'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:30.044978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1225/2459] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:29 2022] DELETE /identity/v3/users/b94cfd04e71e4f69ab4ca766c360ea29 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:30.478675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:30.481167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:30.508549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51309fe2-92fd-4c91-accd-81c873ceadbd admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:30.594058 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:30.595170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:30.595468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:30.595746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] PATH_INFO: `/v3/projects/643aca7e7f194749b2ee55817a35032a` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:30.597184 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643aca7e7f194749b2ee55817a35032a)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:30.599370 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:30.692924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643aca7e7f194749b2ee55817a35032a', 'request_id': 'req-08a3ee13-b800-453e-be18-98d21e7a12e9'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:30.713876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643aca7e7f194749b2ee55817a35032a', 'request_id': 'req-08a3ee13-b800-453e-be18-98d21e7a12e9'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:30.744343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643aca7e7f194749b2ee55817a35032a', 'request_id': 'req-08a3ee13-b800-453e-be18-98d21e7a12e9'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:30.759819 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] The token cache is being invalidate because project 643aca7e7f194749b2ee55817a35032a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:30.760092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:30.760934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-08a3ee13-b800-453e-be18-98d21e7a12e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:30.763179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1235/2460] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:30 2022] DELETE /identity/v3/projects/643aca7e7f194749b2ee55817a35032a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:58:30.768466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e48d-91ef-4b5d-af30-dbafede43522 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:30.769191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e48d-91ef-4b5d-af30-dbafede43522 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:30.769191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e48d-91ef-4b5d-af30-dbafede43522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:30.835893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-8428e48d-91ef-4b5d-af30-dbafede43522 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:30.837367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8428e48d-91ef-4b5d-af30-dbafede43522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:30.960261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1226/2461] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:30.968554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb298e0-aabf-4039-80ee-8437d6f711b6 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:30.969028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb298e0-aabf-4039-80ee-8437d6f711b6 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:30.969417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb298e0-aabf-4039-80ee-8437d6f711b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.000458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ccb298e0-aabf-4039-80ee-8437d6f711b6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:31.001845 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ccb298e0-aabf-4039-80ee-8437d6f711b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:31.011807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1236/2462] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:31.016965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e48d-91ef-4b5d-af30-dbafede43522 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:31.033390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e48d-91ef-4b5d-af30-dbafede43522 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:31.040624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:31.048415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.049402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.050072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.051558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:58:31.052055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:31.055170 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:31.083270 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1227/2463] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:58:31.089511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb298e0-aabf-4039-80ee-8437d6f711b6 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:31.121287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb298e0-aabf-4039-80ee-8437d6f711b6 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:31.156231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:31.156987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.157300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.157584 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.158255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:31.160228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:31.228374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1237/2464] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:58:31 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:31.233051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:31.261162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396075cf-3d0d-4c3f-bbb9-3cbb3aae688d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:31.456954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:31.457878 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.458156 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.458477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.459597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:31.462463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:31.512975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1228/2465] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:58:31 2022] POST /identity/v3/users => generated 287 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:58:31.516136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:31.529285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4847ce-ce8f-4f35-b5a0-99483ee91a6e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:31.537501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:31.537879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.538111 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.538423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.539247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:31.541172 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:31.554251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1238/2466] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:58:31.558499 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:31.572061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ab965-02e7-4f26-9ce5-a74ea2bd7f6f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:31.577890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251c967-7339-4265-b3cb-e81819e89b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:31.578707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c967-7339-4265-b3cb-e81819e89b1e admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.579058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c967-7339-4265-b3cb-e81819e89b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.579339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c967-7339-4265-b3cb-e81819e89b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.580087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c967-7339-4265-b3cb-e81819e89b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:31.582039 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c967-7339-4265-b3cb-e81819e89b1e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:31.601907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1229/2467] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:31.602607 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:31.614383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b1b4-c486-4d6d-8f1d-98b9fcb9045a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:31.619898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:31.620973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.621269 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.621522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] PATH_INFO: `/v3/projects/62bae77a339a47228bbe57976c9c1114/users/2a70af422b204bf081de74a1b967e5a5/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.644327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] RBAC: Authorizing `identity:create_grant(project_id=62bae77a339a47228bbe57976c9c1114, user_id=2a70af422b204bf081de74a1b967e5a5, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:31.650428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:31.673123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1239/2468] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:31 2022] PUT /identity/v3/projects/62bae77a339a47228bbe57976c9c1114/users/2a70af422b204bf081de74a1b967e5a5/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:31.678940 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290cae4-ab2e-49c4-85bd-0e2a25ddf589 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.679295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290cae4-ab2e-49c4-85bd-0e2a25ddf589 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.679784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290cae4-ab2e-49c4-85bd-0e2a25ddf589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:31.705255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d290cae4-ab2e-49c4-85bd-0e2a25ddf589 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:31.707603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d290cae4-ab2e-49c4-85bd-0e2a25ddf589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:31.805913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1230/2469] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:31.820699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:31.826450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:31.973318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804316-6adb-4457-a9bc-cba8578b082f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:31.978140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:31.980913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:31.980913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:31.980913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:32.004127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:32.005726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:32.030384 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1240/2470] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:33.521028 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c254bd-17dc-4d97-80a3-e7d2f9b4da33 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:33.521621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c254bd-17dc-4d97-80a3-e7d2f9b4da33 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:33.522352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c254bd-17dc-4d97-80a3-e7d2f9b4da33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:33.566619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-27c254bd-17dc-4d97-80a3-e7d2f9b4da33 None None] MFA Rules not processed for user `2a70af422b204bf081de74a1b967e5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:33.568286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-27c254bd-17dc-4d97-80a3-e7d2f9b4da33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:33.732740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1231/2471] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:33.736112 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:33.765624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c586c-95d5-4be6-b619-e6b0c3ac704c service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:33.794900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:33.796138 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:33.796706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:33.797325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:33.798728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:33.808401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:33.901330 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1241/2472] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:58:33 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:33.907296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c254bd-17dc-4d97-80a3-e7d2f9b4da33 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:33.935693 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c254bd-17dc-4d97-80a3-e7d2f9b4da33 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:34.060528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:34.061319 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:34.061587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:34.061861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:34.064846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:34.064846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:34.130783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1232/2473] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:58:33 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:34.131189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:34.166500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189f6b-c5bf-4ee7-bf3c-1cc415e71919 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:34.181215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:34.182529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:34.183234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:34.184017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:34.186585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:34.191615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:34.223924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1242/2474] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:34.226376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:34.251542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b25c0f-1614-462e-804f-19fca53157f1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:34.262168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:34.263404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:34.264428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:34.268806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] PATH_INFO: `/v3/projects/d49fae5c9b2b4167a4d53d07582af5a7/users/53f4cd95f44f40be9d77c590ecbd5e64/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:34.286798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49fae5c9b2b4167a4d53d07582af5a7, user_id=53f4cd95f44f40be9d77c590ecbd5e64, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:34.289707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:34.308347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1233/2475] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:34 2022] PUT /identity/v3/projects/d49fae5c9b2b4167a4d53d07582af5a7/users/53f4cd95f44f40be9d77c590ecbd5e64/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:34.314802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:34.327945 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6113f64-e56b-419a-bdf2-ca0f772283df admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:34.423518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b811416-164e-44cb-987e-4eb1cefb4acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:34.424406 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811416-164e-44cb-987e-4eb1cefb4acf admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:34.424856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811416-164e-44cb-987e-4eb1cefb4acf admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:34.425209 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811416-164e-44cb-987e-4eb1cefb4acf admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:34.426429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b811416-164e-44cb-987e-4eb1cefb4acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:34.428980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b811416-164e-44cb-987e-4eb1cefb4acf admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:34.484481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1243/2476] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:34.485945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:34.503062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2eff4c-aaa9-4677-ae6d-0726bbad9ae5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:34.518387 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:34.522265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:34.522265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:34.525077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] PATH_INFO: `/v3/projects/d49fae5c9b2b4167a4d53d07582af5a7/users/53f4cd95f44f40be9d77c590ecbd5e64/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:34.528774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49fae5c9b2b4167a4d53d07582af5a7, user_id=53f4cd95f44f40be9d77c590ecbd5e64, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:34.532245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:34.547790 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1234/2477] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:34 2022] PUT /identity/v3/projects/d49fae5c9b2b4167a4d53d07582af5a7/users/53f4cd95f44f40be9d77c590ecbd5e64/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:58:35.694613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d83019-b132-4630-a189-262eff9d81ab None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:35.697079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d83019-b132-4630-a189-262eff9d81ab None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:35.697653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d83019-b132-4630-a189-262eff9d81ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:35.762767 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-89d83019-b132-4630-a189-262eff9d81ab None None] MFA Rules not processed for user `53f4cd95f44f40be9d77c590ecbd5e64`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:35.764580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-89d83019-b132-4630-a189-262eff9d81ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:35.974102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1244/2478] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:35.994694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:35.997674 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:36.115592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d59827-d031-41d1-9994-20a67a6ec338 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:36.120468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:36.121500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:36.121788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:36.122100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:36.233152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:36.234890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:36.258223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1235/2479] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:37.816105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d83019-b132-4630-a189-262eff9d81ab None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:37.817751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-89d83019-b132-4630-a189-262eff9d81ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:37.831354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d83019-b132-4630-a189-262eff9d81ab None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:37.837704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c241761-9b19-45c3-aea1-2c2d08981d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:37.838752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c241761-9b19-45c3-aea1-2c2d08981d0f service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:37.839271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c241761-9b19-45c3-aea1-2c2d08981d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:37.839811 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c241761-9b19-45c3-aea1-2c2d08981d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:37.854687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c241761-9b19-45c3-aea1-2c2d08981d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:37.856445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c241761-9b19-45c3-aea1-2c2d08981d0f service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:37.871481 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1245/2480] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:42.234099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:42.236277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:42.344909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f392e39-94ae-4361-93f4-b877d5e71701 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:42.353778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:42.356364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:42.356364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:42.356364 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:42.358913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:42.452754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:42.455881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:42.473082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1236/2481] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:58:44.418173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c0430-4988-401b-a0c7-91725ee13df8 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:44.418774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c0430-4988-401b-a0c7-91725ee13df8 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:44.418995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c0430-4988-401b-a0c7-91725ee13df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:44.440975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-725c0430-4988-401b-a0c7-91725ee13df8 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:44.442700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-725c0430-4988-401b-a0c7-91725ee13df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:44.519318 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1246/2482] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 4 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:44.534883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:44.536423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:44.689261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6efb3-880e-4610-9f5f-e834e4f0f591 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:44.696642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:44.697627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:44.698231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:44.698811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:44.709965 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:44.711484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:44.722502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1237/2483] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 14:58:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 04 14:58:51.326093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c0430-4988-401b-a0c7-91725ee13df8 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:51.401061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c0430-4988-401b-a0c7-91725ee13df8 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:51.407110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:51.408429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:51.408786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:51.409187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] PATH_INFO: `/v3/users/2a70af422b204bf081de74a1b967e5a5` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:51.410645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a70af422b204bf081de74a1b967e5a5)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:51.412978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:51.466429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a70af422b204bf081de74a1b967e5a5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.479493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a70af422b204bf081de74a1b967e5a5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.480135 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a70af422b204bf081de74a1b967e5a5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.480831 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a70af422b204bf081de74a1b967e5a5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.490430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a70af422b204bf081de74a1b967e5a5'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.506808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1247/2484] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:51 2022] DELETE /identity/v3/users/2a70af422b204bf081de74a1b967e5a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:51.720681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:51.735148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff37b-8bee-48c5-be08-b4c6709b6cb3 service swift] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:51.808695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:51.809662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:51.809988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:51.810260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] PATH_INFO: `/v3/projects/62bae77a339a47228bbe57976c9c1114` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:51.811543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=62bae77a339a47228bbe57976c9c1114)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:51.813625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:51.876578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62bae77a339a47228bbe57976c9c1114', 'request_id': 'req-29848796-1e2d-437c-90f3-7f758f8b4d5f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.883566 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62bae77a339a47228bbe57976c9c1114', 'request_id': 'req-29848796-1e2d-437c-90f3-7f758f8b4d5f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.889954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62bae77a339a47228bbe57976c9c1114', 'request_id': 'req-29848796-1e2d-437c-90f3-7f758f8b4d5f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.904289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] The token cache is being invalidate because project 62bae77a339a47228bbe57976c9c1114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:51.904600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.905315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:51.907435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1238/2485] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:51 2022] DELETE /identity/v3/projects/62bae77a339a47228bbe57976c9c1114 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:51.912476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:51.914074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:51.933770 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a3aa-7f94-4da4-9f05-0c1bca38e0ed admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.015802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.016787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.017216 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.017886 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] PATH_INFO: `/v3/users/53f4cd95f44f40be9d77c590ecbd5e64` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.019487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f4cd95f44f40be9d77c590ecbd5e64)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:52.022415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:52.086009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f4cd95f44f40be9d77c590ecbd5e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.101179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f4cd95f44f40be9d77c590ecbd5e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.101179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f4cd95f44f40be9d77c590ecbd5e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.101179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f4cd95f44f40be9d77c590ecbd5e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.110374 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b341b920-6007-44d3-863f-156c4c5e5bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f4cd95f44f40be9d77c590ecbd5e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.123443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1248/2486] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:58:51 2022] DELETE /identity/v3/users/53f4cd95f44f40be9d77c590ecbd5e64 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:52.334842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:52.336402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:52.351524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29848796-1e2d-437c-90f3-7f758f8b4d5f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.422352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.423504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.423846 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.424181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] PATH_INFO: `/v3/projects/d49fae5c9b2b4167a4d53d07582af5a7` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.425805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49fae5c9b2b4167a4d53d07582af5a7)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:52.428110 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:52.494131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49fae5c9b2b4167a4d53d07582af5a7', 'request_id': 'req-eb05aa08-a264-4c65-b17e-f9ba3793aa69'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.500665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49fae5c9b2b4167a4d53d07582af5a7', 'request_id': 'req-eb05aa08-a264-4c65-b17e-f9ba3793aa69'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.506988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49fae5c9b2b4167a4d53d07582af5a7', 'request_id': 'req-eb05aa08-a264-4c65-b17e-f9ba3793aa69'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.522753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] The token cache is being invalidate because project d49fae5c9b2b4167a4d53d07582af5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:58:52.523234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.524504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-eb05aa08-a264-4c65-b17e-f9ba3793aa69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:58:52.526734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1239/2487] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:58:52 2022] DELETE /identity/v3/projects/d49fae5c9b2b4167a4d53d07582af5a7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:58:52.533510 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea60ae3-02d8-4307-be56-2acf06495acc None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.534447 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea60ae3-02d8-4307-be56-2acf06495acc None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.534849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea60ae3-02d8-4307-be56-2acf06495acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.554777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-eea60ae3-02d8-4307-be56-2acf06495acc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:52.556978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-eea60ae3-02d8-4307-be56-2acf06495acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:52.626329 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1249/2488] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:52.637214 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33cf99-1e2d-4d51-bfe3-14f101557381 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.637715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33cf99-1e2d-4d51-bfe3-14f101557381 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.638202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33cf99-1e2d-4d51-bfe3-14f101557381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.661284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6a33cf99-1e2d-4d51-bfe3-14f101557381 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:52.662511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6a33cf99-1e2d-4d51-bfe3-14f101557381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:52.672482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1240/2489] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:52.676975 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea60ae3-02d8-4307-be56-2acf06495acc None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:52.687876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea60ae3-02d8-4307-be56-2acf06495acc None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.692820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.693488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.693822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.694094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.694838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:58:52.695182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:52.696918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:52.712596 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1250/2490] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:58:52.713955 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33cf99-1e2d-4d51-bfe3-14f101557381 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:52.724964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33cf99-1e2d-4d51-bfe3-14f101557381 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.730757 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.731888 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.732243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.732564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.733388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:52.735525 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:52.769992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1241/2491] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:58:52 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:52.774797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:52.785401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562168ec-a286-4031-aaa0-d7e23cc85b09 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.848874 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.849479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.849756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.850034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.850748 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:52.852725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:52.875929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1251/2492] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:58:52 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:58:52.883663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:52.903140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bc561-376b-42de-9101-6cbeca174379 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.908785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.909684 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.909975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.910235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.911099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:52.912978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:52.924536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1242/2493] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:52.929035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:52.940087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f15709-7320-4ef2-bc8c-b72c21621342 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.945693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb80e79-0b0f-4466-979b-9cf9fcfb0377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.946399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb80e79-0b0f-4466-979b-9cf9fcfb0377 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.946677 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb80e79-0b0f-4466-979b-9cf9fcfb0377 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.946958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb80e79-0b0f-4466-979b-9cf9fcfb0377 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:52.947723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb80e79-0b0f-4466-979b-9cf9fcfb0377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:52.949531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb80e79-0b0f-4466-979b-9cf9fcfb0377 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:52.962742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1252/2494] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:58:52.967435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:52.978378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5294-4aed-4114-a58b-912c17a2ce53 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:52.983574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:52.984472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:52.984748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:52.985042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] PATH_INFO: `/v3/projects/eebb50667b50443eb7ba05a174e22b9a/users/ba6d02f2fa914658ab14e6fdc99c3e64/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:53.003066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebb50667b50443eb7ba05a174e22b9a, user_id=ba6d02f2fa914658ab14e6fdc99c3e64, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:53.005830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:53.022882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1243/2495] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:58:52 2022] PUT /identity/v3/projects/eebb50667b50443eb7ba05a174e22b9a/users/ba6d02f2fa914658ab14e6fdc99c3e64/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:58:53.033781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b59f5-8b03-4f5b-a149-0da36e534ec3 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:53.034146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b59f5-8b03-4f5b-a149-0da36e534ec3 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:53.034532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b59f5-8b03-4f5b-a149-0da36e534ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:53.066800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d22b59f5-8b03-4f5b-a149-0da36e534ec3 None None] MFA Rules not processed for user `ba6d02f2fa914658ab14e6fdc99c3e64`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:58:53.068265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d22b59f5-8b03-4f5b-a149-0da36e534ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:53.140042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1253/2496] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:58:53.151913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:58:53.154420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:58:53.225931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51172f1b-40e5-47d6-af8a-2f74b918f8f1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:58:53.229953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381fc2f-b537-48bd-a524-34ca61ee3837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:58:53.230691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381fc2f-b537-48bd-a524-34ca61ee3837 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:58:53.230974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381fc2f-b537-48bd-a524-34ca61ee3837 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:58:53.231250 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381fc2f-b537-48bd-a524-34ca61ee3837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:58:53.240628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381fc2f-b537-48bd-a524-34ca61ee3837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:58:53.242095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381fc2f-b537-48bd-a524-34ca61ee3837 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:58:53.250975 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1244/2497] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:58:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:10.995386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b59f5-8b03-4f5b-a149-0da36e534ec3 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:11.074562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b59f5-8b03-4f5b-a149-0da36e534ec3 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:11.079777 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:11.080680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:11.080998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:11.081270 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] PATH_INFO: `/v3/users/ba6d02f2fa914658ab14e6fdc99c3e64` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:11.082479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6d02f2fa914658ab14e6fdc99c3e64)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:11.084407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:11.138768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6d02f2fa914658ab14e6fdc99c3e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:11.152890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6d02f2fa914658ab14e6fdc99c3e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:11.153669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6d02f2fa914658ab14e6fdc99c3e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:11.154289 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6d02f2fa914658ab14e6fdc99c3e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:11.166392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6d02f2fa914658ab14e6fdc99c3e64'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:11.181086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1254/2498] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:10 2022] DELETE /identity/v3/users/ba6d02f2fa914658ab14e6fdc99c3e64 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:11.186318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:11.186924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:11.186924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:11.206296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:11.207550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:11.276327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1245/2499] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:11.296580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:11.298123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:11.363424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab51861-c5b7-4d33-a83d-3e9a1288b381 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:11.367341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef8f70-1e2d-4bbe-a533-5b088084cca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:11.367994 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef8f70-1e2d-4bbe-a533-5b088084cca0 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:11.368283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef8f70-1e2d-4bbe-a533-5b088084cca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:11.368539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef8f70-1e2d-4bbe-a533-5b088084cca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:11.379031 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef8f70-1e2d-4bbe-a533-5b088084cca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:11.380629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef8f70-1e2d-4bbe-a533-5b088084cca0 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:11.391605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1255/2500] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:12.060131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.062154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:12.090354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e0ce-b6cd-4b1e-814a-f82162e395d5 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:12.105972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:12.106647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.106920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.107203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] PATH_INFO: `/v3/projects/eebb50667b50443eb7ba05a174e22b9a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.108694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebb50667b50443eb7ba05a174e22b9a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:12.110787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:12.190991 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebb50667b50443eb7ba05a174e22b9a', 'request_id': 'req-e4614381-a56f-49de-a219-e87662cb240d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:12.198928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebb50667b50443eb7ba05a174e22b9a', 'request_id': 'req-e4614381-a56f-49de-a219-e87662cb240d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:12.205392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebb50667b50443eb7ba05a174e22b9a', 'request_id': 'req-e4614381-a56f-49de-a219-e87662cb240d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:12.234626 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] The token cache is being invalidate because project eebb50667b50443eb7ba05a174e22b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:12.235515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:12.236879 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e4614381-a56f-49de-a219-e87662cb240d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:12.240157 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1246/2501] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:12 2022] DELETE /identity/v3/projects/eebb50667b50443eb7ba05a174e22b9a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:12.245417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa83cd8-0c07-4ae3-9ce4-53ad6eb7c936 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.246455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa83cd8-0c07-4ae3-9ce4-53ad6eb7c936 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.246957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa83cd8-0c07-4ae3-9ce4-53ad6eb7c936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.278385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2aa83cd8-0c07-4ae3-9ce4-53ad6eb7c936 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:12.280055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2aa83cd8-0c07-4ae3-9ce4-53ad6eb7c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:12.360973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1256/2502] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:12.364917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd203ea-52cd-4893-ba6e-004b9525edb7 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.365526 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd203ea-52cd-4893-ba6e-004b9525edb7 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.365871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd203ea-52cd-4893-ba6e-004b9525edb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.391589 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-4bd203ea-52cd-4893-ba6e-004b9525edb7 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:12.393040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4bd203ea-52cd-4893-ba6e-004b9525edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:12.413099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1247/2503] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:12.418688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa83cd8-0c07-4ae3-9ce4-53ad6eb7c936 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.429576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa83cd8-0c07-4ae3-9ce4-53ad6eb7c936 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:12.435456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:12.436487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.436983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.437432 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.438888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:12.439663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:12.441628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:12.460836 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1257/2504] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:12.462481 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd203ea-52cd-4893-ba6e-004b9525edb7 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.471945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd203ea-52cd-4893-ba6e-004b9525edb7 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:12.477321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:12.478318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.479190 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.479537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.480197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:12.482203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:12.511523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1248/2505] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:12 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:12.515978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.525968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f159f3-cf86-4da6-8811-4501ebb6d603 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:12.588904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:12.589473 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.589755 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.590155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.590714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:12.592626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:12.617537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1258/2506] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:12 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:12.619661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.632754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9656d38-d75b-4c99-865e-4a7c789a560b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:12.638572 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:12.639358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.639669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.639947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.640880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:12.642503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:12.654540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1249/2507] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:12.658820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.671206 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625daf7a-dc64-4690-b851-c65b9ea6f165 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:12.677299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68b14d-63ac-44cd-8c66-82def6fb9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:12.677983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68b14d-63ac-44cd-8c66-82def6fb9ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.678274 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68b14d-63ac-44cd-8c66-82def6fb9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.678543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68b14d-63ac-44cd-8c66-82def6fb9ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.679283 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68b14d-63ac-44cd-8c66-82def6fb9ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:12.680906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68b14d-63ac-44cd-8c66-82def6fb9ad8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:12.695805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1259/2508] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:12.697825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.707638 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3b800-51b8-43c9-9bf5-dd52bfe02bf1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:12.712455 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:12.713446 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.713750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.713983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] PATH_INFO: `/v3/projects/27a81782dd804e5db4a63eee6e839138/users/d600e4f5bf704934b286af88a0cf0b8a/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.732005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a81782dd804e5db4a63eee6e839138, user_id=d600e4f5bf704934b286af88a0cf0b8a, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:12.734968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:12.754547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1250/2509] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:12 2022] PUT /identity/v3/projects/27a81782dd804e5db4a63eee6e839138/users/d600e4f5bf704934b286af88a0cf0b8a/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:12.822958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:12.823166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:12.823511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:12.855775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] MFA Rules not processed for user `d600e4f5bf704934b286af88a0cf0b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:12.860169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:12.949239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1260/2510] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:12.958353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:12.960152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:13.034372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5a7a6-a9db-4a21-9e7f-b652fe381906 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:13.038322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:13.038999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:13.039287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:13.039562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:13.049497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:13.050979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:13.064256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1251/2511] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:20.201434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:20.203019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:20.265232 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb3e8a-7f0a-4525-9f9a-55a14aec37d2 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:20.271043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:20.272045 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:20.272456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:20.272768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:20.275986 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:20.348516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:20.350286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:20.362123 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1261/2512] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:21.899318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:21.909067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbe305-3ef5-4f48-a3ca-450c7b930f25 service glance] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:21.913286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:21.913988 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:21.914268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:21.914562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:21.919264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:21.931173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:21.932919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:21.945792 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1252/2513] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:24.569857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:24.572004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:24.591791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09aa0a-c63e-40a8-9784-9f080f797b8e service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:24.597074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:24.597959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:24.598409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:24.598878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] PATH_INFO: `/v3/users/aff6489e5a6441509376c0dca551f03c` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:24.600261 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff6489e5a6441509376c0dca551f03c)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:24.602531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:24.661325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff6489e5a6441509376c0dca551f03c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:24.675703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff6489e5a6441509376c0dca551f03c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:24.676403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff6489e5a6441509376c0dca551f03c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:24.677220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff6489e5a6441509376c0dca551f03c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:24.686256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-ca373bef-c6c8-42b6-803f-6ad8c3e64e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff6489e5a6441509376c0dca551f03c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:24.702729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1262/2514] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:24 2022] DELETE /identity/v3/users/aff6489e5a6441509376c0dca551f03c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:24.896549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:24.914617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd8631-9fd4-47a8-bf6b-f208dd2800f1 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:24.977880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:24.978767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:24.979052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:24.979322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] PATH_INFO: `/v3/projects/35d8071dbd7246228c7a252c40fae038` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:24.980581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d8071dbd7246228c7a252c40fae038)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:24.982758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.045861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d8071dbd7246228c7a252c40fae038', 'request_id': 'req-28ec27af-74f7-4492-9643-8e3473c7dcca'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:25.053748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d8071dbd7246228c7a252c40fae038', 'request_id': 'req-28ec27af-74f7-4492-9643-8e3473c7dcca'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:25.059832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d8071dbd7246228c7a252c40fae038', 'request_id': 'req-28ec27af-74f7-4492-9643-8e3473c7dcca'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:25.074280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] The token cache is being invalidate because project 35d8071dbd7246228c7a252c40fae038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:25.074569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:25.075222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-28ec27af-74f7-4492-9643-8e3473c7dcca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:25.080555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1253/2515] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:24 2022] DELETE /identity/v3/projects/35d8071dbd7246228c7a252c40fae038 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:25.082539 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bfe99-ec16-44e9-af89-ae889bca0b3c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.082899 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bfe99-ec16-44e9-af89-ae889bca0b3c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.083243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bfe99-ec16-44e9-af89-ae889bca0b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.104433 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a19bfe99-ec16-44e9-af89-ae889bca0b3c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:25.105708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a19bfe99-ec16-44e9-af89-ae889bca0b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:25.194934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1263/2516] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:25.197046 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490252-46fd-4a45-a364-69c512dceb4e None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.197549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490252-46fd-4a45-a364-69c512dceb4e None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.197795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97490252-46fd-4a45-a364-69c512dceb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.239472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-97490252-46fd-4a45-a364-69c512dceb4e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:25.240876 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-97490252-46fd-4a45-a364-69c512dceb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:25.249691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1254/2517] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:25.253832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bfe99-ec16-44e9-af89-ae889bca0b3c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:25.263853 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bfe99-ec16-44e9-af89-ae889bca0b3c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:25.268895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:25.269691 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.270067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.270431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.271379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:25.271665 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:25.273404 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.288366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1264/2518] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:25.293469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490252-46fd-4a45-a364-69c512dceb4e None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:25.302437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97490252-46fd-4a45-a364-69c512dceb4e None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:25.311503 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:25.313030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.313460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.313897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.314892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:25.316829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.355665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1255/2519] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:25 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:25.356962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:25.370118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b272f5e-f398-4de9-82ce-b265217c1f01 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:25.434536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:25.435305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.435592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.435869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.436612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:25.438808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.467517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1265/2520] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:25 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:25.472280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:25.497507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b128c-2027-414a-942c-7945bf8fb570 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:25.509423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:25.510484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.511301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.512203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.514210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:25.517135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.544883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1256/2521] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:25.548437 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:25.565727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850f3a7-ce88-4b59-8c21-ec75a0aa7954 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:25.573139 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b498072-cc2f-4818-8fd8-20b28e614159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:25.573862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498072-cc2f-4818-8fd8-20b28e614159 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.574107 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498072-cc2f-4818-8fd8-20b28e614159 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.574380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b498072-cc2f-4818-8fd8-20b28e614159 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.575132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b498072-cc2f-4818-8fd8-20b28e614159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:25.577844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b498072-cc2f-4818-8fd8-20b28e614159 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.591563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1266/2522] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:25.595917 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:25.608195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d50426-476c-4bcf-8c26-23e3806f2725 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:25.628371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:25.629577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.629857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.630131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] PATH_INFO: `/v3/projects/34db6792b8aa4276b793850a9acba35b/users/057547e939734968a47ef7cf7eb0e1c8/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.660641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34db6792b8aa4276b793850a9acba35b, user_id=057547e939734968a47ef7cf7eb0e1c8, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:25.663340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.688932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1257/2523] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:25 2022] PUT /identity/v3/projects/34db6792b8aa4276b793850a9acba35b/users/057547e939734968a47ef7cf7eb0e1c8/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:25.694931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.695248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.695533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.718807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:25.720028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:25.796830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1267/2524] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:25.808012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:25.810483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:25.888942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d13ea-95c3-4b0a-9e5b-df4c18752b95 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:25.893842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:25.894600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:25.894949 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:25.895288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:25.905877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:25.907498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:25.931550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1258/2525] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:26.541785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:26.543592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:26.562681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60aab-c817-4062-9e87-5ec87ec83bfa None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:26.568385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:26.569349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:26.569632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:26.569907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] PATH_INFO: `/v3/users/a0944ce1e8504fb1b50507ace4e63649` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:26.571293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0944ce1e8504fb1b50507ace4e63649)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:26.573334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:26.626158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0944ce1e8504fb1b50507ace4e63649'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:26.640997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0944ce1e8504fb1b50507ace4e63649'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:26.641889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0944ce1e8504fb1b50507ace4e63649'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:26.642632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0944ce1e8504fb1b50507ace4e63649'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:26.652744 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a0e37e8b-73ef-4ec1-8fc9-2f745da74569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0944ce1e8504fb1b50507ace4e63649'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:26.668062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1268/2526] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:26 2022] DELETE /identity/v3/users/a0944ce1e8504fb1b50507ace4e63649 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:26.940960 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:26.951909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d665186-0e75-4462-ba98-787a9a3ef195 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.036916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.037827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.038119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.038395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] PATH_INFO: `/v3/projects/367321a836de445f9e65af1baac99c98` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.039686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=367321a836de445f9e65af1baac99c98)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.041727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.099622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367321a836de445f9e65af1baac99c98', 'request_id': 'req-610334b5-b668-477b-afb0-c5eb7fb3adf4'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:27.106644 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367321a836de445f9e65af1baac99c98', 'request_id': 'req-610334b5-b668-477b-afb0-c5eb7fb3adf4'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:27.112677 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367321a836de445f9e65af1baac99c98', 'request_id': 'req-610334b5-b668-477b-afb0-c5eb7fb3adf4'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:27.130877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] The token cache is being invalidate because project 367321a836de445f9e65af1baac99c98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:27.131229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:27.131871 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-610334b5-b668-477b-afb0-c5eb7fb3adf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:27.133583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1259/2527] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:26 2022] DELETE /identity/v3/projects/367321a836de445f9e65af1baac99c98 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:27.138916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e42d4a-acb0-4807-ba66-0f9f7564e5da None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.139287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e42d4a-acb0-4807-ba66-0f9f7564e5da None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.139680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e42d4a-acb0-4807-ba66-0f9f7564e5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.163136 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-65e42d4a-acb0-4807-ba66-0f9f7564e5da None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:27.164443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-65e42d4a-acb0-4807-ba66-0f9f7564e5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:27.254763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1269/2528] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:27.256645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4656cc-d126-4f65-9a0f-0f6fdc780536 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.256978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4656cc-d126-4f65-9a0f-0f6fdc780536 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.257257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4656cc-d126-4f65-9a0f-0f6fdc780536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.278412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0e4656cc-d126-4f65-9a0f-0f6fdc780536 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:27.279994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0e4656cc-d126-4f65-9a0f-0f6fdc780536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:27.288937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1260/2529] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:27.293950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e42d4a-acb0-4807-ba66-0f9f7564e5da None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.306867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e42d4a-acb0-4807-ba66-0f9f7564e5da None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.312052 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811865a-6dd1-4b3c-81c7-9c6e572f478b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.312764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811865a-6dd1-4b3c-81c7-9c6e572f478b admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.313182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811865a-6dd1-4b3c-81c7-9c6e572f478b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.313429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811865a-6dd1-4b3c-81c7-9c6e572f478b admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.314971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811865a-6dd1-4b3c-81c7-9c6e572f478b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:27.315260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811865a-6dd1-4b3c-81c7-9c6e572f478b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.317006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811865a-6dd1-4b3c-81c7-9c6e572f478b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.335187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1270/2530] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:27.340684 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4656cc-d126-4f65-9a0f-0f6fdc780536 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.359312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4656cc-d126-4f65-9a0f-0f6fdc780536 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.365615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.366522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.367154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.367569 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.368648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.370963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.399915 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.400177 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.400700 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.416754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1261/2531] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/projects => generated 401 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:27.421394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.456229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] MFA Rules not processed for user `057547e939734968a47ef7cf7eb0e1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:27.461132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:27.550379 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a8ed6-c500-4d6d-9bfd-25970ac41bd9 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.558049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.559631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.560005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.560289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.561149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.565812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.589066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1271/2532] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:27.596569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.599615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1262/2533] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/users => generated 301 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:27.600166 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:27.604099 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.619400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e2946-cddf-4dce-a2a2-536b0d74949b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.625237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.626079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.626377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.627540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.628329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.631315 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.635471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157faa3-ac48-42de-91ba-d7b0ab3fe605 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.643134 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1263/2534] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:27.647329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.654155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.654155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.654155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.654155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.654820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.656829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.668916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b4c38-26a6-423a-9009-50835f2159d8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.679424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.680466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.680893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.681246 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.682375 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.684699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.689087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1272/2535] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/projects => generated 403 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:27.693226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.705533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1264/2536] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:27.710474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.724453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4b72e-09e8-43f7-81eb-4d0be770d31e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.774074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86262a-eba7-42f4-a315-5a4751b1b771 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.779518 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.780524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.780908 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.781498 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.782463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.784683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.807016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3d944-0fd1-4d41-9562-b7c157bf30a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.807980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d944-0fd1-4d41-9562-b7c157bf30a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.808287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d944-0fd1-4d41-9562-b7c157bf30a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.808586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3d944-0fd1-4d41-9562-b7c157bf30a5 admin admin] PATH_INFO: `/v3/projects/f6e8c1c4ac6847a8a80dd639bae2b9c1/users/f5c937cd19414daaabdee147d22bbfc3/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.808921 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1273/2537] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/users => generated 308 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:27.813309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.824658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa385971-e207-4ebb-8cf9-208f0aa8a730 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.830463 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.831225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.831763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.832257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.833428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.835467 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.840713 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3d944-0fd1-4d41-9562-b7c157bf30a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e8c1c4ac6847a8a80dd639bae2b9c1, user_id=f5c937cd19414daaabdee147d22bbfc3, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.844044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3d944-0fd1-4d41-9562-b7c157bf30a5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.847734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1274/2538] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:27.852165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.861483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1265/2539] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:27 2022] PUT /identity/v3/projects/f6e8c1c4ac6847a8a80dd639bae2b9c1/users/f5c937cd19414daaabdee147d22bbfc3/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:27.867007 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01de095-4cb3-4454-bf91-61e444d72468 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:27.872877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1315bc8-0ff4-498d-9cb5-480283207264 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.873139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1315bc8-0ff4-498d-9cb5-480283207264 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.873418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1315bc8-0ff4-498d-9cb5-480283207264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.903913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d1315bc8-0ff4-498d-9cb5-480283207264 None None] MFA Rules not processed for user `f5c937cd19414daaabdee147d22bbfc3`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:27.905369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d1315bc8-0ff4-498d-9cb5-480283207264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:27.936966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:27.937875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:27.938155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:27.938424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] PATH_INFO: `/v3/projects/3873056d04bd4be7808a3d539945d40c/users/a7e92ca54cee4026b0fccf88a9493705/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:27.955580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3873056d04bd4be7808a3d539945d40c, user_id=a7e92ca54cee4026b0fccf88a9493705, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:27.958295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:27.985851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1275/2540] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:27 2022] PUT /identity/v3/projects/3873056d04bd4be7808a3d539945d40c/users/a7e92ca54cee4026b0fccf88a9493705/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:27.989901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:27.991797 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1266/2541] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:27.993386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1315bc8-0ff4-498d-9cb5-480283207264 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.009994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1315bc8-0ff4-498d-9cb5-480283207264 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.064470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e14b0e-a18b-4341-99a4-7dbdffc36662 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.078631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.085563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.085944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.087036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.088040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.089840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.096127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.096823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.097200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.097506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.098397 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.100493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.102303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1276/2542] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:28.106580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.118393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4f6b-8792-46be-8fa7-5d078856bdb2 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.124053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.125108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.125465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.125823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] PATH_INFO: `/v3/projects/3873056d04bd4be7808a3d539945d40c/users/a7e92ca54cee4026b0fccf88a9493705/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.128030 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3873056d04bd4be7808a3d539945d40c, user_id=a7e92ca54cee4026b0fccf88a9493705, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.131198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.133003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1267/2543] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:27 2022] POST /identity/v3/projects => generated 399 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:28.137342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.144904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1277/2544] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:28 2022] PUT /identity/v3/projects/3873056d04bd4be7808a3d539945d40c/users/a7e92ca54cee4026b0fccf88a9493705/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:28.228297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3afac-b333-4cde-965e-014a3e3150a5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.317478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.318363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.318733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.319030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.319852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.323272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.355795 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1268/2545] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:28 2022] POST /identity/v3/users => generated 306 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:28.360223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.373985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a38b2-726e-44c5-909e-4f0ec77bdf74 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.380863 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.381817 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.382307 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.382639 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.384132 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.385545 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.404701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.405295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1278/2546] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:28.446181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a27c3-7a89-4383-aa08-2f4889da39e6 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.471136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.472487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.472893 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.473272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] PATH_INFO: `/v3/projects/69de1399d7414b898169f106597118bc/users/8584454e5d1043bfa5186eef07be08f7/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.542221 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=69de1399d7414b898169f106597118bc, user_id=8584454e5d1043bfa5186eef07be08f7, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.545427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.585823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1269/2547] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:28 2022] PUT /identity/v3/projects/69de1399d7414b898169f106597118bc/users/8584454e5d1043bfa5186eef07be08f7/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:28.590243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.605840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ed0fc-a77e-4d20-9b9a-e62bad653315 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.709841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a72811-2791-4f27-b8e6-fbf379e9481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.710526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a72811-2791-4f27-b8e6-fbf379e9481a admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.716213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a72811-2791-4f27-b8e6-fbf379e9481a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.716554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a72811-2791-4f27-b8e6-fbf379e9481a admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.717452 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a72811-2791-4f27-b8e6-fbf379e9481a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.719124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a72811-2791-4f27-b8e6-fbf379e9481a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.738988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1279/2548] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:28.745252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.759868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970c376-e838-4ae6-8c44-56735ee33c9a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:28.766651 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:28.768742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.769296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.769824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] PATH_INFO: `/v3/projects/69de1399d7414b898169f106597118bc/users/8584454e5d1043bfa5186eef07be08f7/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.772449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69de1399d7414b898169f106597118bc, user_id=8584454e5d1043bfa5186eef07be08f7, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:28.779706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:28.797753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1270/2549] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:28 2022] PUT /identity/v3/projects/69de1399d7414b898169f106597118bc/users/8584454e5d1043bfa5186eef07be08f7/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:28.809729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca751f9-dbf1-424e-997c-0bc4cfd43cef None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:28.810409 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca751f9-dbf1-424e-997c-0bc4cfd43cef None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:28.810783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca751f9-dbf1-424e-997c-0bc4cfd43cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:28.847183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-1ca751f9-dbf1-424e-997c-0bc4cfd43cef None None] MFA Rules not processed for user `8584454e5d1043bfa5186eef07be08f7`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:28.848722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1ca751f9-dbf1-424e-997c-0bc4cfd43cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:28.934100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1280/2550] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:28.946750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:28.949687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:29.041149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253b43-61d1-47bc-90d2-8f866d996a74 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:29.048447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:29.049349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:29.049728 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:29.050227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:29.064899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:29.066736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:29.079614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1271/2551] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:29.104059 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24493ae6-c6fc-48d2-a1b9-bc204abd1224 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:29.104860 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24493ae6-c6fc-48d2-a1b9-bc204abd1224 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:29.105512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24493ae6-c6fc-48d2-a1b9-bc204abd1224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:29.141928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-24493ae6-c6fc-48d2-a1b9-bc204abd1224 None None] MFA Rules not processed for user `a7e92ca54cee4026b0fccf88a9493705`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:29.143644 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-24493ae6-c6fc-48d2-a1b9-bc204abd1224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:29.180152 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:29.202077 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb322-80b7-41ee-b813-238cfef10811 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:29.215593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:29.216502 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:29.217002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:29.217445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:29.271636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1281/2552] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:29.334519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:29.335995 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:29.348913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1272/2553] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:30.914102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24493ae6-c6fc-48d2-a1b9-bc204abd1224 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:30.984760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24493ae6-c6fc-48d2-a1b9-bc204abd1224 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:30.989594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:30.990601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:30.991039 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:30.991316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] PATH_INFO: `/v3/users/057547e939734968a47ef7cf7eb0e1c8` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:30.992517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057547e939734968a47ef7cf7eb0e1c8)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:30.994847 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:31.061201 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057547e939734968a47ef7cf7eb0e1c8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.067815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057547e939734968a47ef7cf7eb0e1c8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.068494 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057547e939734968a47ef7cf7eb0e1c8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.069075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057547e939734968a47ef7cf7eb0e1c8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.079521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057547e939734968a47ef7cf7eb0e1c8'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.093015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1282/2554] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:30 2022] DELETE /identity/v3/users/057547e939734968a47ef7cf7eb0e1c8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:31.296511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:31.320602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ded35-3b01-4791-9a0d-7f5daeada065 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:31.412447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:31.413169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:31.413451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:31.413749 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] PATH_INFO: `/v3/projects/34db6792b8aa4276b793850a9acba35b` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:31.415105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=34db6792b8aa4276b793850a9acba35b)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:31.417150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:31.473255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34db6792b8aa4276b793850a9acba35b', 'request_id': 'req-d84b0df6-4a1b-42e5-8698-052293f621bd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.480994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34db6792b8aa4276b793850a9acba35b', 'request_id': 'req-d84b0df6-4a1b-42e5-8698-052293f621bd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.488031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34db6792b8aa4276b793850a9acba35b', 'request_id': 'req-d84b0df6-4a1b-42e5-8698-052293f621bd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.500997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] The token cache is being invalidate because project 34db6792b8aa4276b793850a9acba35b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:31.501504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.502174 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.503706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1273/2555] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:31 2022] DELETE /identity/v3/projects/34db6792b8aa4276b793850a9acba35b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:31.508449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:31.510328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:31.523699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a30bb-bf5e-4b4c-8a64-be78c80df0d9 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:31.583377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:31.584297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:31.584567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:31.584865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] PATH_INFO: `/v3/users/a7e92ca54cee4026b0fccf88a9493705` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:31.586087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e92ca54cee4026b0fccf88a9493705)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:31.588067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:31.642573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e92ca54cee4026b0fccf88a9493705'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.655648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e92ca54cee4026b0fccf88a9493705'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.656380 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e92ca54cee4026b0fccf88a9493705'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.657093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e92ca54cee4026b0fccf88a9493705'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.665890 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6a775da2-de3b-4c26-9f00-28fc3c260015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e92ca54cee4026b0fccf88a9493705'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:31.678567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1283/2556] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:31 2022] DELETE /identity/v3/users/a7e92ca54cee4026b0fccf88a9493705 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:31.874086 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:31.876069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:31.889884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b0df6-4a1b-42e5-8698-052293f621bd admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:31.957445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:31.958391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:31.958671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:31.958948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] PATH_INFO: `/v3/projects/3873056d04bd4be7808a3d539945d40c` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:31.960889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3873056d04bd4be7808a3d539945d40c)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:31.963794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.025736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3873056d04bd4be7808a3d539945d40c', 'request_id': 'req-b1b879cc-a802-474f-9b68-ac443ba93305'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:32.032200 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3873056d04bd4be7808a3d539945d40c', 'request_id': 'req-b1b879cc-a802-474f-9b68-ac443ba93305'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:32.038341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3873056d04bd4be7808a3d539945d40c', 'request_id': 'req-b1b879cc-a802-474f-9b68-ac443ba93305'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:32.052070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] The token cache is being invalidate because project 3873056d04bd4be7808a3d539945d40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:32.052413 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:32.053074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b1b879cc-a802-474f-9b68-ac443ba93305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:32.054632 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1274/2557] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:31 2022] DELETE /identity/v3/projects/3873056d04bd4be7808a3d539945d40c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:32.060246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869db86-048d-4079-b5a2-e3f0c9c3acb9 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.060732 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869db86-048d-4079-b5a2-e3f0c9c3acb9 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.060875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869db86-048d-4079-b5a2-e3f0c9c3acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.079295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4869db86-048d-4079-b5a2-e3f0c9c3acb9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:32.080493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4869db86-048d-4079-b5a2-e3f0c9c3acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:32.146255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1284/2558] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:32.154323 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e96e9-a99d-43c1-b424-709a0da031a2 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.154608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e96e9-a99d-43c1-b424-709a0da031a2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.154887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e96e9-a99d-43c1-b424-709a0da031a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.175062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-d08e96e9-a99d-43c1-b424-709a0da031a2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:32.176295 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d08e96e9-a99d-43c1-b424-709a0da031a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:32.185718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1275/2559] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:32.190448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869db86-048d-4079-b5a2-e3f0c9c3acb9 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.201664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869db86-048d-4079-b5a2-e3f0c9c3acb9 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.206750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.207474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.207782 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.208061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.208855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:32.209161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.210891 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.223497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1285/2560] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:32.228292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e96e9-a99d-43c1-b424-709a0da031a2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.239376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e96e9-a99d-43c1-b424-709a0da031a2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.247223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.248434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.248807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.249169 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.249945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.251922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.282953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1276/2561] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:32 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:32.287941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.303128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57d88-5627-48f2-b149-fb7bb80ff18a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.392849 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.394033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.394721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.395336 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.396073 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.398124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.423426 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1286/2562] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:32 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:32.428709 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.443548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958fd3-7d84-459c-81d7-a28917639971 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.448451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.449247 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.449586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.449956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.450841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.452500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.468581 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1277/2563] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:32.470254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.480542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c140a2-033c-4198-8946-71384b049f2c admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.486167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be60dc-b1b3-43c3-9998-a89090ea1c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.486943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be60dc-b1b3-43c3-9998-a89090ea1c21 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.487222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be60dc-b1b3-43c3-9998-a89090ea1c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.487487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be60dc-b1b3-43c3-9998-a89090ea1c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.488256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be60dc-b1b3-43c3-9998-a89090ea1c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.490167 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be60dc-b1b3-43c3-9998-a89090ea1c21 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.507125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1287/2564] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:32.508372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.520640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecdcf5-9bc2-4b0b-b3b6-13c086fd35d4 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.525550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.526633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.526967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.527342 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] PATH_INFO: `/v3/projects/57078fd144594c0d82ae638a8ea5a631/users/c639c81dc43d427aa0e9962ad1fb6f44/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.547118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57078fd144594c0d82ae638a8ea5a631, user_id=c639c81dc43d427aa0e9962ad1fb6f44, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.549961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.565947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1278/2565] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:32 2022] PUT /identity/v3/projects/57078fd144594c0d82ae638a8ea5a631/users/c639c81dc43d427aa0e9962ad1fb6f44/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:32.579742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2ff9-e21a-47cc-8b91-b838aa91492b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.580122 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2ff9-e21a-47cc-8b91-b838aa91492b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.580457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2ff9-e21a-47cc-8b91-b838aa91492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.615730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d91e2ff9-e21a-47cc-8b91-b838aa91492b None None] MFA Rules not processed for user `c639c81dc43d427aa0e9962ad1fb6f44`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:32.617602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d91e2ff9-e21a-47cc-8b91-b838aa91492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:32.695585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.697357 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:32.717311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1288/2566] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:32.722347 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e2ff9-e21a-47cc-8b91-b838aa91492b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.765185 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15672b5-2a6b-4331-8ee0-63e1f9ac7f37 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.769411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.770234 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.770497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.770838 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.772831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:32.806715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e2ff9-e21a-47cc-8b91-b838aa91492b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.812383 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.813109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.813398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.813678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.814414 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.816388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.845757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1289/2567] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:32 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:32.850155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.850423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.852220 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.864337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1279/2568] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:32.919682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b4027-5fe4-4140-b42b-acc9a4f59f48 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.924977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.926002 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.926303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.926714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.927325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.929386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.953995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1290/2569] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:32 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:32.955643 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:32.965780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be132698-d8b0-46c0-9ec5-783ae0476ff6 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:32.971286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:32.971957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:32.972233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:32.972495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:32.973263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:32.974855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:32.986062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1280/2570] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:32.990353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:33.001707 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d86b57-1dd8-4ffd-83bf-64e1a8ef99dd admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:33.007466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:33.008458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:33.008724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:33.009047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] PATH_INFO: `/v3/projects/1d26724a702b461a982ea5af96bed781/users/9084adee55b048f68f24931b8adeeb15/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:33.027418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d26724a702b461a982ea5af96bed781, user_id=9084adee55b048f68f24931b8adeeb15, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:33.030311 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:33.049792 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1291/2571] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:32 2022] PUT /identity/v3/projects/1d26724a702b461a982ea5af96bed781/users/9084adee55b048f68f24931b8adeeb15/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:33.054982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:33.071882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c538709-3348-403f-a75c-3bed091ea6bf admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:33.145824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cd69f-9e3e-4ee6-b2d7-ec0cb050b423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:33.146557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cd69f-9e3e-4ee6-b2d7-ec0cb050b423 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:33.146850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cd69f-9e3e-4ee6-b2d7-ec0cb050b423 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:33.147131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cd69f-9e3e-4ee6-b2d7-ec0cb050b423 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:33.147923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cd69f-9e3e-4ee6-b2d7-ec0cb050b423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:33.149698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cd69f-9e3e-4ee6-b2d7-ec0cb050b423 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:33.166928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1281/2572] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:33.168394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:33.181328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061bab8-99fd-47a4-9ac2-6273158a98df admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:33.186522 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:33.187495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:33.187766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:33.188033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] PATH_INFO: `/v3/projects/1d26724a702b461a982ea5af96bed781/users/9084adee55b048f68f24931b8adeeb15/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:33.190465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d26724a702b461a982ea5af96bed781, user_id=9084adee55b048f68f24931b8adeeb15, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:33.193325 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:33.205210 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1292/2573] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:33 2022] PUT /identity/v3/projects/1d26724a702b461a982ea5af96bed781/users/9084adee55b048f68f24931b8adeeb15/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:33.222060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34de8b-3524-417f-bafa-1a3cc58006d1 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:33.222340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34de8b-3524-417f-bafa-1a3cc58006d1 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:33.222611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34de8b-3524-417f-bafa-1a3cc58006d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:33.256161 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ab34de8b-3524-417f-bafa-1a3cc58006d1 None None] MFA Rules not processed for user `9084adee55b048f68f24931b8adeeb15`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:33.257905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ab34de8b-3524-417f-bafa-1a3cc58006d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:33.334539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1282/2574] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:33 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:33.347196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:33.349203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:33.414373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298a57c-26c3-48f2-adc8-6a6a5c1de0fd admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:33.418441 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:33.419196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:33.419642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:33.420067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:33.433276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:33.434905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:33.446647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1293/2575] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:33 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:33.529678 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34de8b-3524-417f-bafa-1a3cc58006d1 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:33.541240 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34de8b-3524-417f-bafa-1a3cc58006d1 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:33.545640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc69a6b-79b3-4a28-a6ef-932184e448ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:33.546514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc69a6b-79b3-4a28-a6ef-932184e448ce service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:33.547072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc69a6b-79b3-4a28-a6ef-932184e448ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:33.547376 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc69a6b-79b3-4a28-a6ef-932184e448ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:33.621883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc69a6b-79b3-4a28-a6ef-932184e448ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:33.623805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc69a6b-79b3-4a28-a6ef-932184e448ce service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:33.635754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1283/2576] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:33.824456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:33.826079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:33.915977 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f137de9-158c-4492-a50c-8b8a68a640c6 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:33.923537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:33.924477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:33.924918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:33.925229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] PATH_INFO: `/v3/users/f5c937cd19414daaabdee147d22bbfc3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:33.926641 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c937cd19414daaabdee147d22bbfc3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:33.929250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:33.991972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c937cd19414daaabdee147d22bbfc3'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:34.007183 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c937cd19414daaabdee147d22bbfc3'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:34.008029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c937cd19414daaabdee147d22bbfc3'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:34.008613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c937cd19414daaabdee147d22bbfc3'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:34.017999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c937cd19414daaabdee147d22bbfc3'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:34.037747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1294/2577] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:33 2022] DELETE /identity/v3/users/f5c937cd19414daaabdee147d22bbfc3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:34.043300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:34.043729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:34.044092 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:34.068103 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:34.069584 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:34.164178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1284/2578] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:34.178573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:34.181598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:34.265074 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2f5ed-41e0-4c99-ae85-8395b3250faf admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:34.270460 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:34.271248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:34.271603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:34.271856 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:34.285187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:34.286711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:34.301647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1295/2579] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:34.835985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:34.837699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:34.876118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37f70c-3b79-4aaf-8933-17e2eabc9281 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:34.884761 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:34.886218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:34.886611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:34.886969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] PATH_INFO: `/v3/projects/f6e8c1c4ac6847a8a80dd639bae2b9c1` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:34.888530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e8c1c4ac6847a8a80dd639bae2b9c1)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:34.893705 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:34.987675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e8c1c4ac6847a8a80dd639bae2b9c1', 'request_id': 'req-d30392d9-7c59-431a-872f-3e26f37bc9de'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.000527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e8c1c4ac6847a8a80dd639bae2b9c1', 'request_id': 'req-d30392d9-7c59-431a-872f-3e26f37bc9de'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.008571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e8c1c4ac6847a8a80dd639bae2b9c1', 'request_id': 'req-d30392d9-7c59-431a-872f-3e26f37bc9de'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.038184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] The token cache is being invalidate because project f6e8c1c4ac6847a8a80dd639bae2b9c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:35.038687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.039775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.042831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1285/2580] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:34 2022] DELETE /identity/v3/projects/f6e8c1c4ac6847a8a80dd639bae2b9c1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:35.047271 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:35.063305 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702f9e-c45f-452a-906a-eefc5b68cb38 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:35.142562 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:35.147079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:35.148108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:35.149339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] PATH_INFO: `/v3/users/8584454e5d1043bfa5186eef07be08f7` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:35.152222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8584454e5d1043bfa5186eef07be08f7)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:35.155652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:35.246914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8584454e5d1043bfa5186eef07be08f7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.269233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8584454e5d1043bfa5186eef07be08f7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.271985 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8584454e5d1043bfa5186eef07be08f7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.273067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8584454e5d1043bfa5186eef07be08f7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.286247 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-74c2623d-d1ef-4d2d-99b9-f9feda19d6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8584454e5d1043bfa5186eef07be08f7'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.318966 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1296/2581] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:35 2022] DELETE /identity/v3/users/8584454e5d1043bfa5186eef07be08f7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:35.716732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:35.719788 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:35.777136 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30392d9-7c59-431a-872f-3e26f37bc9de admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:35.878997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:35.880004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:35.880637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:35.880959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] PATH_INFO: `/v3/projects/69de1399d7414b898169f106597118bc` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:35.882732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] RBAC: Authorizing `identity:delete_project(project_id=69de1399d7414b898169f106597118bc)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:35.885140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:35.967587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69de1399d7414b898169f106597118bc', 'request_id': 'req-2a03575f-dbff-4756-abad-ecb2244bc22c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.975916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69de1399d7414b898169f106597118bc', 'request_id': 'req-2a03575f-dbff-4756-abad-ecb2244bc22c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:35.984125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69de1399d7414b898169f106597118bc', 'request_id': 'req-2a03575f-dbff-4756-abad-ecb2244bc22c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:36.000018 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] The token cache is being invalidate because project 69de1399d7414b898169f106597118bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:36.000317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:36.001101 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2a03575f-dbff-4756-abad-ecb2244bc22c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:36.003114 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1286/2582] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:35 2022] DELETE /identity/v3/projects/69de1399d7414b898169f106597118bc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:36.010155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265d19-138f-418d-b53c-c169352a0a7f None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.010595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265d19-138f-418d-b53c-c169352a0a7f None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.010971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265d19-138f-418d-b53c-c169352a0a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.031350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0a265d19-138f-418d-b53c-c169352a0a7f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:36.032764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0a265d19-138f-418d-b53c-c169352a0a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:36.104423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1297/2583] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:36.113052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46104936-6a19-444f-aa66-5fcadf753e48 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.113348 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46104936-6a19-444f-aa66-5fcadf753e48 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.113812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46104936-6a19-444f-aa66-5fcadf753e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.136080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-46104936-6a19-444f-aa66-5fcadf753e48 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:36.137380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-46104936-6a19-444f-aa66-5fcadf753e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:36.150276 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1287/2584] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:36.151360 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a265d19-138f-418d-b53c-c169352a0a7f None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.162251 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a265d19-138f-418d-b53c-c169352a0a7f None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.167356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.168028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.168282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.168671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.169429 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:36.169745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.171461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:36.185315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1298/2585] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:36.190060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46104936-6a19-444f-aa66-5fcadf753e48 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.200180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46104936-6a19-444f-aa66-5fcadf753e48 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.206398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.207199 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.207551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.207905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.208703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.210703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:36.243683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1288/2586] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:36 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:36.248182 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.270354 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9efdb-8ea8-40ca-a8c5-afdc380b04c8 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.356099 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.356513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.356812 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.357096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.357795 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.359706 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:36.384717 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1299/2587] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:36 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:36.386541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.397353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e01ff-3926-4589-be60-0830d70f06f0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.402308 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.402980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.403454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.403732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.404465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.406150 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:36.417467 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1289/2588] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:36.422791 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.434187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5943cd-7748-43b1-8b46-051e9a939532 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.439279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cae14c-ba9b-4a44-8655-44afdef5d108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.439968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cae14c-ba9b-4a44-8655-44afdef5d108 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.440243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cae14c-ba9b-4a44-8655-44afdef5d108 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.440523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cae14c-ba9b-4a44-8655-44afdef5d108 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.441300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cae14c-ba9b-4a44-8655-44afdef5d108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.442934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cae14c-ba9b-4a44-8655-44afdef5d108 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:36.458789 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1300/2589] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:36.460020 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.472602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c57132-4551-4e75-8792-99049158db54 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.477818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.478729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.479027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.479299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] PATH_INFO: `/v3/projects/90d536e996584da58c58e3f12bdd68ed/users/c49526a1c1394d87a2fb9a4897532730/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.495019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d536e996584da58c58e3f12bdd68ed, user_id=c49526a1c1394d87a2fb9a4897532730, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.498159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:36.513952 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1290/2590] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:36 2022] PUT /identity/v3/projects/90d536e996584da58c58e3f12bdd68ed/users/c49526a1c1394d87a2fb9a4897532730/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:36.525446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655daf3-da64-4645-899e-2513b4dfa2a8 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.525711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655daf3-da64-4645-899e-2513b4dfa2a8 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.525996 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655daf3-da64-4645-899e-2513b4dfa2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.556611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b655daf3-da64-4645-899e-2513b4dfa2a8 None None] MFA Rules not processed for user `c49526a1c1394d87a2fb9a4897532730`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:36.559072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b655daf3-da64-4645-899e-2513b4dfa2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:36.674158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1301/2591] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:36.678759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.798016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2d03b-9229-49e5-b917-03ab09c0e4ea admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.812449 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.812831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.813115 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.813393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.814106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.816074 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:36.868109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1291/2592] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:36 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:36.872381 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655daf3-da64-4645-899e-2513b4dfa2a8 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:36.899595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655daf3-da64-4645-899e-2513b4dfa2a8 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:36.983822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:36.984529 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:36.985303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:36.985303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:36.985914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:36.992399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:37.023800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1302/2593] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:36 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:37.028209 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:37.063059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d0eef-3ae3-43f4-8b20-4d27721bf9ef admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:37.080922 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:37.081873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.082305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.082729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.083827 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:37.088887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:37.115157 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1292/2594] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:37.119527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:37.134957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713db95-b68a-4d8b-996c-162b10c0d5db admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:37.140316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:37.141444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.141825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.142173 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] PATH_INFO: `/v3/projects/fa749ef4a25341eb8e99286b6ed216c8/users/0d2891aa27b845b2883c291389e9dca1/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.163302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa749ef4a25341eb8e99286b6ed216c8, user_id=0d2891aa27b845b2883c291389e9dca1, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:37.167205 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:37.189991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1303/2595] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:37 2022] PUT /identity/v3/projects/fa749ef4a25341eb8e99286b6ed216c8/users/0d2891aa27b845b2883c291389e9dca1/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:37.195119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:37.219509 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab177a-e949-4b87-b278-79df55f18a50 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:37.343225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:37.345798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:37.382286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:37.383015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.383261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.383707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.384445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:37.386195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:37.406318 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1293/2596] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:37.407571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:37.418527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd61935-d2b6-42bb-a342-a0ce7c1532a0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:37.423980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ade667-66eb-4e2d-96a0-cc77e4cfacea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:37.425111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ade667-66eb-4e2d-96a0-cc77e4cfacea admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.425392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ade667-66eb-4e2d-96a0-cc77e4cfacea admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.425696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ade667-66eb-4e2d-96a0-cc77e4cfacea admin admin] PATH_INFO: `/v3/projects/fa749ef4a25341eb8e99286b6ed216c8/users/0d2891aa27b845b2883c291389e9dca1/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.431138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ade667-66eb-4e2d-96a0-cc77e4cfacea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa749ef4a25341eb8e99286b6ed216c8, user_id=0d2891aa27b845b2883c291389e9dca1, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:37.433887 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ade667-66eb-4e2d-96a0-cc77e4cfacea admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:37.435526 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04297c-802c-4a27-aba4-66e7f3b5f378 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:37.441576 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:37.442564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.442844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.443115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] PATH_INFO: `/v3/users/da47bf4c34f243bb9493d8bc11891cef` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.444363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da47bf4c34f243bb9493d8bc11891cef)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:37.446458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:37.446884 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1294/2597] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:37 2022] PUT /identity/v3/projects/fa749ef4a25341eb8e99286b6ed216c8/users/0d2891aa27b845b2883c291389e9dca1/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:37.458916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.459462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.460292 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.508895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] MFA Rules not processed for user `0d2891aa27b845b2883c291389e9dca1`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:37.510177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:37.512895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da47bf4c34f243bb9493d8bc11891cef'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:37.524999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da47bf4c34f243bb9493d8bc11891cef'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:37.525714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da47bf4c34f243bb9493d8bc11891cef'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:37.526324 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da47bf4c34f243bb9493d8bc11891cef'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:37.535604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da47bf4c34f243bb9493d8bc11891cef'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:37.550699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1304/2598] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:37 2022] DELETE /identity/v3/users/da47bf4c34f243bb9493d8bc11891cef => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:37.605800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1295/2599] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:37.617438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:37.618884 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:37.696014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e52e78-c4fa-43cb-9e97-0df1294df953 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:37.700619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba6ff9-3b8c-4f47-bb90-4d30aad7b7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:37.701309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6ff9-3b8c-4f47-bb90-4d30aad7b7cf service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.701593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6ff9-3b8c-4f47-bb90-4d30aad7b7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.701870 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba6ff9-3b8c-4f47-bb90-4d30aad7b7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.764650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:37.770860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:37.789106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba6ff9-3b8c-4f47-bb90-4d30aad7b7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:37.791564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba6ff9-3b8c-4f47-bb90-4d30aad7b7cf service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:37.805934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1305/2600] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:37.862760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb755f-f49a-46c4-9281-f45df0b4bc1d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:37.942973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:37.944552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:37.945094 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:37.945575 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] PATH_INFO: `/v3/projects/f512333663064960847a1dd30c177cf9` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:37.947736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f512333663064960847a1dd30c177cf9)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:37.950078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.018635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f512333663064960847a1dd30c177cf9', 'request_id': 'req-1ee539ef-b3ad-4b82-93b9-a945ee54f140'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.025277 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f512333663064960847a1dd30c177cf9', 'request_id': 'req-1ee539ef-b3ad-4b82-93b9-a945ee54f140'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.031817 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f512333663064960847a1dd30c177cf9', 'request_id': 'req-1ee539ef-b3ad-4b82-93b9-a945ee54f140'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.045997 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] The token cache is being invalidate because project f512333663064960847a1dd30c177cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:38.046278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.046916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-1ee539ef-b3ad-4b82-93b9-a945ee54f140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.048462 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1296/2601] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:37 2022] DELETE /identity/v3/projects/f512333663064960847a1dd30c177cf9 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:38.053517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55658164-df6f-4663-a094-db95c0dbaab2 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.053808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55658164-df6f-4663-a094-db95c0dbaab2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.054091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55658164-df6f-4663-a094-db95c0dbaab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.073105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-55658164-df6f-4663-a094-db95c0dbaab2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:38.074355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-55658164-df6f-4663-a094-db95c0dbaab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:38.158398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1306/2602] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:38.166697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe491c87-b9e7-4237-ab14-7610eac261be None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.166986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe491c87-b9e7-4237-ab14-7610eac261be None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.167444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe491c87-b9e7-4237-ab14-7610eac261be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.186906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-fe491c87-b9e7-4237-ab14-7610eac261be None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:38.188151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fe491c87-b9e7-4237-ab14-7610eac261be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:38.198210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1297/2603] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:38.205387 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55658164-df6f-4663-a094-db95c0dbaab2 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.218367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55658164-df6f-4663-a094-db95c0dbaab2 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.224682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.225386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.225696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.225949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.226698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:38.226982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.228583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.244907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1307/2604] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:38.247617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe491c87-b9e7-4237-ab14-7610eac261be None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.258299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe491c87-b9e7-4237-ab14-7610eac261be None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.263798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.264452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.264730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.265038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.265729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.267613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.296627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1298/2605] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:38.297919 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.308081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182957c-527f-4358-ac6f-e96441cb3948 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.379978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.381086 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.381458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.381798 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.382603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.384652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.405933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1308/2606] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:38.410740 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.421798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa6770-1d61-452a-ac3e-a98567396174 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.426857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.427618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.427996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.428336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.429178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.430981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.435238 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.437390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:38.441752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1299/2607] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:38.446204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.450257 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e508b45-0ea3-4ee7-b70b-9e86c84c7d73 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.455056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.456044 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e502f16-e84c-4bd2-865d-70bf8aaafcd2 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.456244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.456559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.456956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] PATH_INFO: `/v3/users/c49526a1c1394d87a2fb9a4897532730` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.458219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49526a1c1394d87a2fb9a4897532730)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.460307 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.460782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.461506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.461850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.462187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.462973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.465243 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.475202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1300/2608] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:38.479264 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.489754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfe2fa-8e6b-4d70-bdd6-631ba798c9db admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.495227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930df5d8-b3dd-4168-96f8-32d2faa99ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.496107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df5d8-b3dd-4168-96f8-32d2faa99ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.496384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df5d8-b3dd-4168-96f8-32d2faa99ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.496682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930df5d8-b3dd-4168-96f8-32d2faa99ca8 admin admin] PATH_INFO: `/v3/projects/612e8e967124405aacad538206e9a46a/users/9ffe216847784500be318cd87c3ed3fa/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.507829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49526a1c1394d87a2fb9a4897532730'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.512472 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df5d8-b3dd-4168-96f8-32d2faa99ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=612e8e967124405aacad538206e9a46a, user_id=9ffe216847784500be318cd87c3ed3fa, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.516535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930df5d8-b3dd-4168-96f8-32d2faa99ca8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.521192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49526a1c1394d87a2fb9a4897532730'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.522018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49526a1c1394d87a2fb9a4897532730'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.522631 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49526a1c1394d87a2fb9a4897532730'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.529916 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-0fffdec4-3996-4246-ae2d-6ceb7f0f3ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49526a1c1394d87a2fb9a4897532730'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:38.530252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1301/2609] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:38 2022] PUT /identity/v3/projects/612e8e967124405aacad538206e9a46a/users/9ffe216847784500be318cd87c3ed3fa/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:38.540861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.541204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.541444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.542933 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1309/2610] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:38 2022] DELETE /identity/v3/users/c49526a1c1394d87a2fb9a4897532730 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:38.547458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.547746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.548019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.564942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:38.566574 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:38.567068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] MFA Rules not processed for user `9ffe216847784500be318cd87c3ed3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:38.568224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:38.623585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1310/2611] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:38.630009 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1302/2612] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:38.633657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.634414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.635050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:38.635681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:38.648535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84817ac9-4ed3-4928-b43c-41a6d4b7f9c9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.653352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.654100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.654420 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.654729 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.655501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.657434 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.685881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1303/2613] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:38.687052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.692200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a4899-4028-48b7-9908-5c9e0581da31 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.742093 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.742876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.743158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.743451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.748466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadd3a0-72a2-4a0c-a967-8c30161e9a87 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.752455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.753283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.753939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.754055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.754219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.754487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.755483 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.757429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.763645 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1311/2614] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:38.781181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1304/2615] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:38 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:38.782237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.791563 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97898d-eb56-4b29-8de8-183187b6fed1 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.796642 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.797366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.797687 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.798019 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.798950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.800556 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.811255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1312/2616] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:38.815798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.826868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47654c-253d-45e1-b458-82ffc099ec2a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.832549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.833488 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.833908 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.834256 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] PATH_INFO: `/v3/projects/c600b4e6baf14c3ca2a31b9c902d3269/users/db4691c0f0184aa695b11bd05d562578/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.851107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c600b4e6baf14c3ca2a31b9c902d3269, user_id=db4691c0f0184aa695b11bd05d562578, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.853911 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.869585 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1305/2617] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:38 2022] PUT /identity/v3/projects/c600b4e6baf14c3ca2a31b9c902d3269/users/db4691c0f0184aa695b11bd05d562578/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:38.873516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.884785 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e7812-5b8e-413d-9e42-efc0d00e9b3e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.942924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ed757-efd7-43fe-9913-189511149553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.943515 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ed757-efd7-43fe-9913-189511149553 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.943821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ed757-efd7-43fe-9913-189511149553 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.944095 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ed757-efd7-43fe-9913-189511149553 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.944844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ed757-efd7-43fe-9913-189511149553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.946457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ed757-efd7-43fe-9913-189511149553 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.957088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1313/2618] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:38.961444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:38.972935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d4ded-3f6a-462d-a4e3-3526a701e08a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:38.978288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:38.979241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:38.979528 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:38.979802 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] PATH_INFO: `/v3/projects/c600b4e6baf14c3ca2a31b9c902d3269/users/db4691c0f0184aa695b11bd05d562578/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:38.982919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c600b4e6baf14c3ca2a31b9c902d3269, user_id=db4691c0f0184aa695b11bd05d562578, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:38.986100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:38.997191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1306/2619] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:38 2022] PUT /identity/v3/projects/c600b4e6baf14c3ca2a31b9c902d3269/users/db4691c0f0184aa695b11bd05d562578/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:39.007938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.008301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.008577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.036733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] MFA Rules not processed for user `db4691c0f0184aa695b11bd05d562578`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:39.038092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.049312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:39.051297 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.113293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1314/2620] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:39.123299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fcf3f-dab1-41b2-abd2-faadee17980b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:39.135234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:39.135234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.144531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:39.146280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.146818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.148050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] PATH_INFO: `/v3/projects/90d536e996584da58c58e3f12bdd68ed` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.150325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90d536e996584da58c58e3f12bdd68ed)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:39.157612 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:39.198667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119c759-984e-4327-a666-cb71e0abadcb None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:39.205408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90d536e996584da58c58e3f12bdd68ed', 'request_id': 'req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.213042 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90d536e996584da58c58e3f12bdd68ed', 'request_id': 'req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.221139 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90d536e996584da58c58e3f12bdd68ed', 'request_id': 'req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.236658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] The token cache is being invalidate because project 90d536e996584da58c58e3f12bdd68ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:39.236963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.237697 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.239226 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1307/2621] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:39 2022] DELETE /identity/v3/projects/90d536e996584da58c58e3f12bdd68ed => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:39.243953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:39.245601 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.254967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:39.256626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.256626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.256626 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.258351 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.321301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf5a5e-3904-4232-b0b7-b9d05fdf106c admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:39.322479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:39.323988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:39.326701 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:39.327700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.327983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.328255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] PATH_INFO: `/v3/users/0d2891aa27b845b2883c291389e9dca1` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.329554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2891aa27b845b2883c291389e9dca1)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:39.331642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:39.334988 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1315/2622] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:39.392284 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2891aa27b845b2883c291389e9dca1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.393034 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2891aa27b845b2883c291389e9dca1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.393690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2891aa27b845b2883c291389e9dca1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.402852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2891aa27b845b2883c291389e9dca1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.421784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2891aa27b845b2883c291389e9dca1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.438058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1308/2623] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:39 2022] DELETE /identity/v3/users/0d2891aa27b845b2883c291389e9dca1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:39.728861 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:39.751023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd612365-f5f9-4f8d-9087-8dcb04cf9662 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:39.776837 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:39.778398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.810931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:39.811939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.812222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.812497 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] PATH_INFO: `/v3/projects/fa749ef4a25341eb8e99286b6ed216c8` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.813751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa749ef4a25341eb8e99286b6ed216c8)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:39.815813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:39.841905 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ff27-ee5c-45d1-b7bd-879bdc80f367 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:39.847683 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:39.848633 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.848927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.849212 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] PATH_INFO: `/v3/users/9ffe216847784500be318cd87c3ed3fa` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.850432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ffe216847784500be318cd87c3ed3fa)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:39.852337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:39.862997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa749ef4a25341eb8e99286b6ed216c8', 'request_id': 'req-21c80cd8-8860-4621-b0df-a1a9f3609a5a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.869949 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa749ef4a25341eb8e99286b6ed216c8', 'request_id': 'req-21c80cd8-8860-4621-b0df-a1a9f3609a5a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.875508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa749ef4a25341eb8e99286b6ed216c8', 'request_id': 'req-21c80cd8-8860-4621-b0df-a1a9f3609a5a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.887800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] The token cache is being invalidate because project fa749ef4a25341eb8e99286b6ed216c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:39.888120 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.888887 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-21c80cd8-8860-4621-b0df-a1a9f3609a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.890787 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1316/2624] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:39 2022] DELETE /identity/v3/projects/fa749ef4a25341eb8e99286b6ed216c8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:39.895636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032409e-537f-4011-a9f1-ae8deedb02a2 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.896040 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032409e-537f-4011-a9f1-ae8deedb02a2 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.896366 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032409e-537f-4011-a9f1-ae8deedb02a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.900117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ffe216847784500be318cd87c3ed3fa'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.900780 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ffe216847784500be318cd87c3ed3fa'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.901427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ffe216847784500be318cd87c3ed3fa'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.908744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ffe216847784500be318cd87c3ed3fa'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.920493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6032409e-537f-4011-a9f1-ae8deedb02a2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:39.920670 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2af500d9-4d94-4c9e-ac1b-0646d26839de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ffe216847784500be318cd87c3ed3fa'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:39.921763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6032409e-537f-4011-a9f1-ae8deedb02a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.934941 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1309/2625] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:39 2022] DELETE /identity/v3/users/9ffe216847784500be318cd87c3ed3fa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:39.939676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a76007-4e32-4cd1-89c0-2687b394dcaf None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.939959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a76007-4e32-4cd1-89c0-2687b394dcaf None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.940230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a76007-4e32-4cd1-89c0-2687b394dcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:39.957804 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b4a76007-4e32-4cd1-89c0-2687b394dcaf None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:39.959033 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b4a76007-4e32-4cd1-89c0-2687b394dcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:39.983763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1317/2626] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:39.991228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:39.991489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:39.991768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.010264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:40.011451 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:40.019022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1310/2627] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:40.022115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1318/2628] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:40.026196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a76007-4e32-4cd1-89c0-2687b394dcaf None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.031239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.032652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:40.036080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a76007-4e32-4cd1-89c0-2687b394dcaf None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.042596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.043336 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.043645 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.043939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.044733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:40.045127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.046933 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.065356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1311/2629] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:59:40.066996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.078825 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e8105-50ed-4372-aad9-0e06521d49ea admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.083602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.084298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.084619 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.084937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.085666 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.087591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.094946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc965f-8046-4c4d-af99-e77be84a4da0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.099169 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.100056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.100339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.100611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.111240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.112708 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.115806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1312/2630] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:40 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:40.119829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.123231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1319/2631] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:40.189689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b75fb-e66a-4a00-b350-745f48ad2a20 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.196306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.197154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.197423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.197695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.198421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.200338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.225628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1313/2632] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:40 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:40.230321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.240693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58383379-3055-4569-8077-cc19621ab516 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.247382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.248062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.248342 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.248617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.249396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.251036 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.264297 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1320/2633] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:40.267938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.278388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1761a-3c43-4e56-9671-f8ecabdc6437 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.283418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcace802-34a4-4fa1-b955-2d61b686314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.284062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcace802-34a4-4fa1-b955-2d61b686314d admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.284350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcace802-34a4-4fa1-b955-2d61b686314d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.284763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcace802-34a4-4fa1-b955-2d61b686314d admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.285530 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcace802-34a4-4fa1-b955-2d61b686314d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.287149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcace802-34a4-4fa1-b955-2d61b686314d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.301999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1314/2634] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:40.303807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.315716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053479bb-ad11-4fc0-bbe5-bdd71c962a34 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.321925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.323352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.323352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.323736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] PATH_INFO: `/v3/projects/626acbe7a5664b5c932afaced8b9d800/users/158c0f48f9c74b269a7d53e5b2a09282/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.344397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] RBAC: Authorizing `identity:create_grant(project_id=626acbe7a5664b5c932afaced8b9d800, user_id=158c0f48f9c74b269a7d53e5b2a09282, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.347272 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.363309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1321/2635] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:40 2022] PUT /identity/v3/projects/626acbe7a5664b5c932afaced8b9d800/users/158c0f48f9c74b269a7d53e5b2a09282/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:40.375747 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41cd6f-c1d7-4bf2-b984-a8c4ade57228 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.376043 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41cd6f-c1d7-4bf2-b984-a8c4ade57228 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.376494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41cd6f-c1d7-4bf2-b984-a8c4ade57228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.410803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-fc41cd6f-c1d7-4bf2-b984-a8c4ade57228 None None] MFA Rules not processed for user `158c0f48f9c74b269a7d53e5b2a09282`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:40.411968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-fc41cd6f-c1d7-4bf2-b984-a8c4ade57228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:40.441995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.443965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:40.528913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1315/2636] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:40.529100 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489ef44-8854-414e-a44b-8ec8a194844d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.534287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41cd6f-c1d7-4bf2-b984-a8c4ade57228 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.540651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.542105 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.542466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.543172 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] PATH_INFO: `/v3/projects/612e8e967124405aacad538206e9a46a` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.547060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=612e8e967124405aacad538206e9a46a)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.549669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.552064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41cd6f-c1d7-4bf2-b984-a8c4ade57228 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.557208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.557881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.558174 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.558439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.559173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.561549 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.595822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1316/2637] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:40 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:40.601047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.623407 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612e8e967124405aacad538206e9a46a', 'request_id': 'req-2e7b6547-7450-44d1-8b45-1e457867fda2'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.633623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612e8e967124405aacad538206e9a46a', 'request_id': 'req-2e7b6547-7450-44d1-8b45-1e457867fda2'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.642230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612e8e967124405aacad538206e9a46a', 'request_id': 'req-2e7b6547-7450-44d1-8b45-1e457867fda2'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.656696 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] The token cache is being invalidate because project 612e8e967124405aacad538206e9a46a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:40.657096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.657764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.659413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1322/2638] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:40 2022] DELETE /identity/v3/projects/612e8e967124405aacad538206e9a46a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:40.663602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.665286 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:40.676257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0266e6d-e843-455c-8f99-36303e4ef058 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.751111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.753008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.753324 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.753603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.754382 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.756744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.762208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b6547-7450-44d1-8b45-1e457867fda2 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.769523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.770458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.770736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.771092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] PATH_INFO: `/v3/users/db4691c0f0184aa695b11bd05d562578` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.772246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4691c0f0184aa695b11bd05d562578)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.774231 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.784553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1317/2639] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:40 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:40.786536 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.789071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:40.803783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2922515-56ac-4de6-993f-6bc721acbb47 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.810000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.810821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.811224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.811700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.812591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.814522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.826031 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1318/2640] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:40.830405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.843196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab914de-ea45-4724-a054-a6d6eb219c71 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:40.845364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4691c0f0184aa695b11bd05d562578'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.851716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:40.851716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:40.851716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:40.851716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] PATH_INFO: `/v3/projects/7e0c54c6a94f4e168e7ee350f20e08a7/users/70cd934cda6f4d789f9837079735693f/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:40.867993 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0c54c6a94f4e168e7ee350f20e08a7, user_id=70cd934cda6f4d789f9837079735693f, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:40.870695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:40.877493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4691c0f0184aa695b11bd05d562578'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.878255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4691c0f0184aa695b11bd05d562578'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.879108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4691c0f0184aa695b11bd05d562578'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.886222 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1319/2641] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:40 2022] PUT /identity/v3/projects/7e0c54c6a94f4e168e7ee350f20e08a7/users/70cd934cda6f4d789f9837079735693f/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:40.890062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4691c0f0184aa695b11bd05d562578'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:40.890272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:40.892040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:40.911857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1323/2642] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:40 2022] DELETE /identity/v3/users/db4691c0f0184aa695b11bd05d562578 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:40.979333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735005bb-66f6-40f0-9413-0d3241f066b1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.037026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960d79d-40de-49de-b5ae-c90f15d1ea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.037703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d79d-40de-49de-b5ae-c90f15d1ea64 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.037981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d79d-40de-49de-b5ae-c90f15d1ea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.038254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d79d-40de-49de-b5ae-c90f15d1ea64 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.038998 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960d79d-40de-49de-b5ae-c90f15d1ea64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.040606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960d79d-40de-49de-b5ae-c90f15d1ea64 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.050658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1320/2643] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:41.055179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.073416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff90b5-0cf9-4d07-a972-db57992bb604 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.080616 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.081603 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.081883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.082591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] PATH_INFO: `/v3/projects/7e0c54c6a94f4e168e7ee350f20e08a7/users/70cd934cda6f4d789f9837079735693f/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.084763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0c54c6a94f4e168e7ee350f20e08a7, user_id=70cd934cda6f4d789f9837079735693f, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.087459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.099012 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1324/2644] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:41 2022] PUT /identity/v3/projects/7e0c54c6a94f4e168e7ee350f20e08a7/users/70cd934cda6f4d789f9837079735693f/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:41.109441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.109996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.110194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.136473 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] MFA Rules not processed for user `70cd934cda6f4d789f9837079735693f`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:41.137852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.195165 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.197077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.209538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1321/2645] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:41.223267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.225021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.282390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a14e-a7c6-4c58-a4ba-d0f6266ea0f7 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.287229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.288098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.288382 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.288658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] PATH_INFO: `/v3/projects/c600b4e6baf14c3ca2a31b9c902d3269` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.289913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c600b4e6baf14c3ca2a31b9c902d3269)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.291746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427dd56-c6a8-45e3-a29a-6b376d544452 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.291868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.295418 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1095ff-a97d-49f0-8558-6c0719159d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.296061 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1095ff-a97d-49f0-8558-6c0719159d64 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.296339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1095ff-a97d-49f0-8558-6c0719159d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.296616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1095ff-a97d-49f0-8558-6c0719159d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.307254 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1095ff-a97d-49f0-8558-6c0719159d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.309409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1095ff-a97d-49f0-8558-6c0719159d64 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.323159 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1322/2646] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:41.343618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c600b4e6baf14c3ca2a31b9c902d3269', 'request_id': 'req-65e0771d-ec87-439c-ab86-df170fc0d3eb'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:41.353989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c600b4e6baf14c3ca2a31b9c902d3269', 'request_id': 'req-65e0771d-ec87-439c-ab86-df170fc0d3eb'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:41.360088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c600b4e6baf14c3ca2a31b9c902d3269', 'request_id': 'req-65e0771d-ec87-439c-ab86-df170fc0d3eb'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:41.371634 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] The token cache is being invalidate because project c600b4e6baf14c3ca2a31b9c902d3269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:41.371993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:41.372788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:41.374378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1325/2647] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:41 2022] DELETE /identity/v3/projects/c600b4e6baf14c3ca2a31b9c902d3269 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:41.380543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee515a15-5dea-400f-b3ca-8c0a9f5475b6 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.380862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee515a15-5dea-400f-b3ca-8c0a9f5475b6 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.381145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee515a15-5dea-400f-b3ca-8c0a9f5475b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.398196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.400044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.400210 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-ee515a15-5dea-400f-b3ca-8c0a9f5475b6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:41.401265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ee515a15-5dea-400f-b3ca-8c0a9f5475b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.456399 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0771d-ec87-439c-ab86-df170fc0d3eb admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.460213 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.460889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.461117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1323/2648] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:41.461471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.461750 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.463569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.470675 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f4aee-3919-4e19-b169-d9af843cefb2 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.471054 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f4aee-3919-4e19-b169-d9af843cefb2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.471280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f4aee-3919-4e19-b169-d9af843cefb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.490433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-af5f4aee-3919-4e19-b169-d9af843cefb2 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:41.491571 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-af5f4aee-3919-4e19-b169-d9af843cefb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.499561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1324/2649] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:41.503850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f4aee-3919-4e19-b169-d9af843cefb2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.514930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f4aee-3919-4e19-b169-d9af843cefb2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.521468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.522361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.522699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.522999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.524062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:41.524447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.526799 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.530781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.532244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.545080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1325/2650] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 14:59:41.545428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1326/2651] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:41.547244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.559637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356947-cb2d-4d5d-8a90-615a49c835f1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.565904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.568181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.568181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.568181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.568181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.569853 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.599302 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1326/2652] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:41 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:41.604868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.616764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7f230-48e9-46c5-95f3-c7f5b4b06733 service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.688191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.689051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.689395 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.689833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.690533 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.693104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.725725 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1327/2653] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:41 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:41.727863 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.748635 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2646d-bbe8-42ed-bd85-db79a9a6deb8 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.764128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.764128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.764128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.764128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.764783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.764783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.780072 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1327/2654] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:41.781392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.795200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0040d4b-de6f-46db-8e1b-45122ee54111 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.800400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.801175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.801521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.801803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.802579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.804188 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.817026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1328/2655] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:41.822133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.836999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848076da-54de-4a57-a5f4-f75e206889db admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.842541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9005be1-6ac8-4a48-9b07-11372540f6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.843487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005be1-6ac8-4a48-9b07-11372540f6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.843768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005be1-6ac8-4a48-9b07-11372540f6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.844076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9005be1-6ac8-4a48-9b07-11372540f6b1 admin admin] PATH_INFO: `/v3/projects/92c3c374a5c74ccd98ae39ce638f935c/users/09d13c9c45934cb4b392e563ae4a90af/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.861924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:41.863943 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.865469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9005be1-6ac8-4a48-9b07-11372540f6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c3c374a5c74ccd98ae39ce638f935c, user_id=09d13c9c45934cb4b392e563ae4a90af, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.869412 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9005be1-6ac8-4a48-9b07-11372540f6b1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:41.885411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1f3f-cda1-44d2-b50a-ce2718a45e1e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:41.892353 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1328/2656] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:41 2022] PUT /identity/v3/projects/92c3c374a5c74ccd98ae39ce638f935c/users/09d13c9c45934cb4b392e563ae4a90af/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:41.902886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd8fdc-c82a-441c-8233-6f075f0383dd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.903812 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd8fdc-c82a-441c-8233-6f075f0383dd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.904551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd8fdc-c82a-441c-8233-6f075f0383dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.943248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-08bd8fdc-c82a-441c-8233-6f075f0383dd None None] MFA Rules not processed for user `09d13c9c45934cb4b392e563ae4a90af`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:41.945227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-08bd8fdc-c82a-441c-8233-6f075f0383dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:41.981035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:41.982085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:41.982375 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:41.982652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] PATH_INFO: `/v3/users/158c0f48f9c74b269a7d53e5b2a09282` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:41.984015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=158c0f48f9c74b269a7d53e5b2a09282)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:41.987997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.033951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1329/2657] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 14:59:42.039312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd8fdc-c82a-441c-8233-6f075f0383dd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.045566 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158c0f48f9c74b269a7d53e5b2a09282'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.051552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd8fdc-c82a-441c-8233-6f075f0383dd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.058187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.059087 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.059579 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.060017 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.060931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.062260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158c0f48f9c74b269a7d53e5b2a09282'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.063295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158c0f48f9c74b269a7d53e5b2a09282'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.063774 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.064227 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158c0f48f9c74b269a7d53e5b2a09282'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.075179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-1ddca6f2-faf0-4cfe-8bc7-b451144c2d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158c0f48f9c74b269a7d53e5b2a09282'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.098763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1329/2658] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:41 2022] DELETE /identity/v3/users/158c0f48f9c74b269a7d53e5b2a09282 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:42.104862 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1330/2659] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:42 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:42.105958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.106303 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.106602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.106984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.107826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.126697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:42.127898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.183347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8591a4-5293-4e41-a23b-0028e6563f98 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.188291 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.189105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.189390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.189661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.190367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.192299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.214484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1330/2660] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:42.220004 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1331/2661] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:42 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:42.224378 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.224575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.225850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.240060 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26af49-f813-49fa-a8d1-639c421395ef admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.249959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.250763 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.251053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.251359 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.252248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.254616 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.271279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1332/2662] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:42.272700 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.284339 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20a649-c2a6-441b-81a5-3ae5f62ea3d4 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.289797 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.290699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.291000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.291219 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] PATH_INFO: `/v3/projects/fbe4b9e4101340e7b98b74edb80b7371/users/8de6d13796644acfa4f7e645cad65c84/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.296629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285592f6-9d3d-46aa-a064-e76de1c4d0e5 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.300403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.301131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.301419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.301692 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.312734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.314196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.325310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe4b9e4101340e7b98b74edb80b7371, user_id=8de6d13796644acfa4f7e645cad65c84, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.325736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1331/2663] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:42.329259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.346860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1333/2664] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:42 2022] PUT /identity/v3/projects/fbe4b9e4101340e7b98b74edb80b7371/users/8de6d13796644acfa4f7e645cad65c84/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:42.351427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.368236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e5cce-b0d0-4f47-8ebe-d43c0321c520 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.460745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab876ce-d6b1-4001-a951-3c19bbb9cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.461775 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab876ce-d6b1-4001-a951-3c19bbb9cb27 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.462671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab876ce-d6b1-4001-a951-3c19bbb9cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.463063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab876ce-d6b1-4001-a951-3c19bbb9cb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.464590 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab876ce-d6b1-4001-a951-3c19bbb9cb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.466864 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab876ce-d6b1-4001-a951-3c19bbb9cb27 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.480842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1332/2665] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:42.484734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.503858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619c42-ccb9-48e1-a104-159f702662ee admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.510461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.511578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.512035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.512461 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] PATH_INFO: `/v3/projects/fbe4b9e4101340e7b98b74edb80b7371/users/8de6d13796644acfa4f7e645cad65c84/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.515841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe4b9e4101340e7b98b74edb80b7371, user_id=8de6d13796644acfa4f7e645cad65c84, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.519263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.532298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1334/2666] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:42 2022] PUT /identity/v3/projects/fbe4b9e4101340e7b98b74edb80b7371/users/8de6d13796644acfa4f7e645cad65c84/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:42.543237 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.543588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.543892 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.572185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] MFA Rules not processed for user `8de6d13796644acfa4f7e645cad65c84`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:42.573427 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.652580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1333/2667] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:42.652811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.653947 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.660428 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.661968 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.719956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c46f05-9add-44da-902c-f1d3f5825154 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.723699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.724346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.724629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.724932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.726445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ce5c3-d708-43f9-a4ca-781f34ad717b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.731851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.732746 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.733059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.733335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] PATH_INFO: `/v3/projects/626acbe7a5664b5c932afaced8b9d800` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.734538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] RBAC: Authorizing `identity:delete_project(project_id=626acbe7a5664b5c932afaced8b9d800)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.734751 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.736192 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.736423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.746326 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1334/2668] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:42.777514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.786823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626acbe7a5664b5c932afaced8b9d800', 'request_id': 'req-105f6642-bac1-4447-929a-19516090a27e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.792632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1741621-e425-48db-9754-db41f5aed60a service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.793151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626acbe7a5664b5c932afaced8b9d800', 'request_id': 'req-105f6642-bac1-4447-929a-19516090a27e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.798815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626acbe7a5664b5c932afaced8b9d800', 'request_id': 'req-105f6642-bac1-4447-929a-19516090a27e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.811608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] The token cache is being invalidate because project 626acbe7a5664b5c932afaced8b9d800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:42.812143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.812776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.814370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1335/2669] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:42 2022] DELETE /identity/v3/projects/626acbe7a5664b5c932afaced8b9d800 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:42.815127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.817279 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.853268 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.854161 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.854444 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.854721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] PATH_INFO: `/v3/users/09d13c9c45934cb4b392e563ae4a90af` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.856034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d13c9c45934cb4b392e563ae4a90af)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.858170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.882153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f6642-bac1-4447-929a-19516090a27e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:42.888335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:42.889777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:42.890244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:42.890642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] PATH_INFO: `/v3/users/d600e4f5bf704934b286af88a0cf0b8a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:42.892523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d600e4f5bf704934b286af88a0cf0b8a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:42.895310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:42.909772 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d13c9c45934cb4b392e563ae4a90af'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.923802 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d13c9c45934cb4b392e563ae4a90af'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.925614 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d13c9c45934cb4b392e563ae4a90af'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.926304 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d13c9c45934cb4b392e563ae4a90af'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.935514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d13c9c45934cb4b392e563ae4a90af'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.949103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1335/2670] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:42 2022] DELETE /identity/v3/users/09d13c9c45934cb4b392e563ae4a90af => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:42.950103 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:42.951549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:42.955023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd600e4f5bf704934b286af88a0cf0b8a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.955937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd600e4f5bf704934b286af88a0cf0b8a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.957026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd600e4f5bf704934b286af88a0cf0b8a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.972038 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd600e4f5bf704934b286af88a0cf0b8a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:42.986762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2ba3fc7b-ad8a-48b3-bee0-ac81ccdbb0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd600e4f5bf704934b286af88a0cf0b8a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.004143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1336/2671] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:42 2022] DELETE /identity/v3/users/d600e4f5bf704934b286af88a0cf0b8a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:43.006665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ad986-75df-407c-8507-d1cc0a4746fe None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.006970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ad986-75df-407c-8507-d1cc0a4746fe None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.007293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ad986-75df-407c-8507-d1cc0a4746fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.025312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-791ad986-75df-407c-8507-d1cc0a4746fe None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:43.026524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-791ad986-75df-407c-8507-d1cc0a4746fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.030091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f195b9-5712-4b1f-86c8-b10b6b3e3637 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:43.088627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:43.089491 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.089728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.089999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] PATH_INFO: `/v3/users/70cd934cda6f4d789f9837079735693f` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.091473 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70cd934cda6f4d789f9837079735693f)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:43.093470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:43.098948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1337/2672] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:43.100162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.100447 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.100723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.120841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:43.122166 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.131669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1338/2673] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:43.136414 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:43.145626 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.157768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70cd934cda6f4d789f9837079735693f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.170275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70cd934cda6f4d789f9837079735693f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.171033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70cd934cda6f4d789f9837079735693f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.171657 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70cd934cda6f4d789f9837079735693f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.184745 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70cd934cda6f4d789f9837079735693f'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.198759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1336/2674] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:42 2022] DELETE /identity/v3/users/70cd934cda6f4d789f9837079735693f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:43.200003 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:43.201553 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.208992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dcfcb-9d1b-40a5-8e4b-193ef64e9a11 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:43.262927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:43.265637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.265637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.265637 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.266268 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.275538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09b09b-5665-4954-b502-5380f9cc4a63 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:43.281811 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ed28e-5a36-4bdd-b6de-fee912f601ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:43.282477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ed28e-5a36-4bdd-b6de-fee912f601ca service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.282788 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ed28e-5a36-4bdd-b6de-fee912f601ca service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.283028 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ed28e-5a36-4bdd-b6de-fee912f601ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.285294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-be3ed28e-5a36-4bdd-b6de-fee912f601ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.361521 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:43.363332 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ed28e-5a36-4bdd-b6de-fee912f601ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:43.363487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:43.364727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ed28e-5a36-4bdd-b6de-fee912f601ca service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:43.374771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1339/2675] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:43.387204 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1337/2676] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:43.537515 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:43.540828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.565730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220c021-8177-4406-9f19-8ba243cff38b service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:43.575163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:43.576111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.576335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.576615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] PATH_INFO: `/v3/projects/7e0c54c6a94f4e168e7ee350f20e08a7` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.577935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0c54c6a94f4e168e7ee350f20e08a7)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:43.580019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:43.646874 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0c54c6a94f4e168e7ee350f20e08a7', 'request_id': 'req-e7f4b4cb-fb22-4263-aafb-c200f795535e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.655698 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0c54c6a94f4e168e7ee350f20e08a7', 'request_id': 'req-e7f4b4cb-fb22-4263-aafb-c200f795535e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.663615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0c54c6a94f4e168e7ee350f20e08a7', 'request_id': 'req-e7f4b4cb-fb22-4263-aafb-c200f795535e'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.710529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] The token cache is being invalidate because project 7e0c54c6a94f4e168e7ee350f20e08a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:43.710939 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.711745 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.730208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1340/2677] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:43 2022] DELETE /identity/v3/projects/7e0c54c6a94f4e168e7ee350f20e08a7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:43.732678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.733233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.733688 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.765475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:43.767987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.861123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:43.863829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.913903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1338/2678] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:43.913903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:43.913903 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:43.922666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7342b6-0e75-4685-b7c6-82a0dc9b176c None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:43.927763 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:43.928826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:43.929449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:43.929833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] PATH_INFO: `/v3/projects/27a81782dd804e5db4a63eee6e839138` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:43.931990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a81782dd804e5db4a63eee6e839138)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:43.934373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:43.982714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b4cb-fb22-4263-aafb-c200f795535e admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:43.986997 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a81782dd804e5db4a63eee6e839138', 'request_id': 'req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:43.998612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a81782dd804e5db4a63eee6e839138', 'request_id': 'req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.006537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a81782dd804e5db4a63eee6e839138', 'request_id': 'req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.019913 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] The token cache is being invalidate because project 27a81782dd804e5db4a63eee6e839138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:44.020196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.020912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-3286acfa-9be9-4ccd-b45d-9e4d7329a27b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.022768 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1339/2679] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:43 2022] DELETE /identity/v3/projects/27a81782dd804e5db4a63eee6e839138 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:44.026292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0406fab-2bf3-42c0-b9c6-58ab51593ed4 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.026850 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0406fab-2bf3-42c0-b9c6-58ab51593ed4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.027129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0406fab-2bf3-42c0-b9c6-58ab51593ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.049835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.050082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-c0406fab-2bf3-42c0-b9c6-58ab51593ed4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:44.050826 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.051118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.051393 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] PATH_INFO: `/v3/projects/92c3c374a5c74ccd98ae39ce638f935c` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.051834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-c0406fab-2bf3-42c0-b9c6-58ab51593ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.053187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c3c374a5c74ccd98ae39ce638f935c)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.055019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.061032 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1340/2680] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:44.062458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2df7f-3504-4709-8f0b-886b96f6039e None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.062749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2df7f-3504-4709-8f0b-886b96f6039e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.063102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2df7f-3504-4709-8f0b-886b96f6039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.082934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-65d2df7f-3504-4709-8f0b-886b96f6039e None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:44.084760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-65d2df7f-3504-4709-8f0b-886b96f6039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.095345 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1341/2681] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:44.096144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2df7f-3504-4709-8f0b-886b96f6039e None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.108061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2df7f-3504-4709-8f0b-886b96f6039e None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.114133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c3c374a5c74ccd98ae39ce638f935c', 'request_id': 'req-db26c485-3d69-4602-adec-4d4dcc9e71fd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.120786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c3c374a5c74ccd98ae39ce638f935c', 'request_id': 'req-db26c485-3d69-4602-adec-4d4dcc9e71fd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.127023 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c3c374a5c74ccd98ae39ce638f935c', 'request_id': 'req-db26c485-3d69-4602-adec-4d4dcc9e71fd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.141478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] The token cache is being invalidate because project 92c3c374a5c74ccd98ae39ce638f935c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:44.141814 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.142803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-db26c485-3d69-4602-adec-4d4dcc9e71fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.144565 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1341/2682] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:43 2022] DELETE /identity/v3/projects/92c3c374a5c74ccd98ae39ce638f935c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:44.145768 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.146051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.146321 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.164432 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:44.165803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.167499 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.168222 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.168565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.168927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.169666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:44.169972 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.171615 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.174007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1342/2683] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:44.175294 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.176358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.184033 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1342/2684] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:44.184901 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.189507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acd9bf-cf17-4e81-a12a-6feac17705bd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.193816 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.194891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.195179 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.195457 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] PATH_INFO: `/v3/users/8de6d13796644acfa4f7e645cad65c84` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.196536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd706fa-3478-4367-979c-4440eadff2ea admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.196737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de6d13796644acfa4f7e645cad65c84)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.198706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.201595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.202244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.202527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.202803 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.203528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:44.203839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.205548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.218492 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1343/2685] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:44.219296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.221038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.235196 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274a041-7275-4f23-bee1-6bdf4a79a5f9 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.240511 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.241061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.241343 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.241613 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.242361 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.244377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.249040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de6d13796644acfa4f7e645cad65c84'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.249819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de6d13796644acfa4f7e645cad65c84'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.250459 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de6d13796644acfa4f7e645cad65c84'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.258311 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de6d13796644acfa4f7e645cad65c84'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.270707 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de6d13796644acfa4f7e645cad65c84'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.275077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1344/2686] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/projects => generated 359 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:44.275077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.277043 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.288443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1343/2687] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:44 2022] DELETE /identity/v3/users/8de6d13796644acfa4f7e645cad65c84 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:44.289298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.290652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.348542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4dc50-5e01-4dbe-af9e-22d2407036ee admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.375154 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05d8a0-27a2-4f1b-85f1-8370610c255f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.383781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.384588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.385496 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.385634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.386891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.390636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.423821 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1344/2688] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/users => generated 280 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:44.429299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.432072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.432705 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.433075 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.433353 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.434223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.436430 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.441442 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa02e83-39e9-4067-a4e6-78fc7a8fab09 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.448771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.449800 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.450650 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.451162 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.452299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.454244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.467667 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1345/2689] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:44.472298 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.485553 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d81ba9-5ed8-49a9-8a5f-395a4d22d456 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.490179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1345/2690] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:44.492250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.552193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.553005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.553930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.553930 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.554892 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.556786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.568850 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1346/2691] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:44.570486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a212f161-4ab2-4349-8707-39444bfed51e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.573160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.576248 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.577537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.577537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.577537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.578296 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.580293 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.589108 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebc093-fad8-4535-98e0-a122ed4da0a4 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.594652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.595601 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.595904 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.596213 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] PATH_INFO: `/v3/projects/6238a65284054cc0b1524bd9645b0912/users/7489dcefd5e7476c957a00c7e980d0f1/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.606578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1346/2692] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:44.607840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.615344 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6238a65284054cc0b1524bd9645b0912, user_id=7489dcefd5e7476c957a00c7e980d0f1, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.618263 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.621255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747ee4f-b884-4936-a01f-c5539dad87f7 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.627710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd2b27-c20f-4dca-858d-3a309ab476e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.628484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd2b27-c20f-4dca-858d-3a309ab476e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.628904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd2b27-c20f-4dca-858d-3a309ab476e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.629294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd2b27-c20f-4dca-858d-3a309ab476e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.630453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd2b27-c20f-4dca-858d-3a309ab476e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.632109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd2b27-c20f-4dca-858d-3a309ab476e1 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.635406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1347/2693] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:44 2022] PUT /identity/v3/projects/6238a65284054cc0b1524bd9645b0912/users/7489dcefd5e7476c957a00c7e980d0f1/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:44.636563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.638400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.652875 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1347/2694] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:44.654449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4deb33-9572-4900-8918-b50fedcbba2a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.654760 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4deb33-9572-4900-8918-b50fedcbba2a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.655097 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4deb33-9572-4900-8918-b50fedcbba2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.682144 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8e4deb33-9572-4900-8918-b50fedcbba2a None None] MFA Rules not processed for user `7489dcefd5e7476c957a00c7e980d0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:44.683365 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8e4deb33-9572-4900-8918-b50fedcbba2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.707611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab70ccb-4592-4950-ad15-a03ce6e21c16 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.712652 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.713554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.713835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.714102 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] PATH_INFO: `/v3/projects/fbe4b9e4101340e7b98b74edb80b7371` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.715368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbe4b9e4101340e7b98b74edb80b7371)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.717374 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.747175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1348/2695] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:44.748228 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4deb33-9572-4900-8918-b50fedcbba2a None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.766092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4deb33-9572-4900-8918-b50fedcbba2a None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.767720 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbe4b9e4101340e7b98b74edb80b7371', 'request_id': 'req-62575d73-0d5f-4c27-8009-314ab0f2e981'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.777058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbe4b9e4101340e7b98b74edb80b7371', 'request_id': 'req-62575d73-0d5f-4c27-8009-314ab0f2e981'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.782686 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbe4b9e4101340e7b98b74edb80b7371', 'request_id': 'req-62575d73-0d5f-4c27-8009-314ab0f2e981'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.795731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] The token cache is being invalidate because project fbe4b9e4101340e7b98b74edb80b7371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:44.796015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.796676 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:44.798259 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1348/2696] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:44 2022] DELETE /identity/v3/projects/fbe4b9e4101340e7b98b74edb80b7371 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:44.803595 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.804973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.825346 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f7549-4b3c-48ab-a2b6-e2333fd91244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.826017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f7549-4b3c-48ab-a2b6-e2333fd91244 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.826301 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f7549-4b3c-48ab-a2b6-e2333fd91244 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.826578 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f7549-4b3c-48ab-a2b6-e2333fd91244 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.827315 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f7549-4b3c-48ab-a2b6-e2333fd91244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.828973 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f7549-4b3c-48ab-a2b6-e2333fd91244 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.839929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1349/2697] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:44.844470 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.844749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.845047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.863114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:44.863349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62575d73-0d5f-4c27-8009-314ab0f2e981 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.864331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.866849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0806e-c10e-4364-8a3b-be843a7099e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.867577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0806e-c10e-4364-8a3b-be843a7099e2 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.868005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0806e-c10e-4364-8a3b-be843a7099e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.868405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0806e-c10e-4364-8a3b-be843a7099e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.870186 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-53a0806e-c10e-4364-8a3b-be843a7099e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.872883 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1350/2698] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:44.873961 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:44.875230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.887698 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a462f77-17ec-4a75-bf2f-1c6185540318 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:44.893128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce2711-8f9d-46b0-b6da-97e288c139b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:44.894025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce2711-8f9d-46b0-b6da-97e288c139b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.894300 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce2711-8f9d-46b0-b6da-97e288c139b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.894571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce2711-8f9d-46b0-b6da-97e288c139b5 admin admin] PATH_INFO: `/v3/projects/35023f0e4a0842778e3125cb024562c3/users/f322d73b5c5d41d69e197a767d7aea75/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.910950 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce2711-8f9d-46b0-b6da-97e288c139b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35023f0e4a0842778e3125cb024562c3, user_id=f322d73b5c5d41d69e197a767d7aea75, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.913957 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce2711-8f9d-46b0-b6da-97e288c139b5 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.928050 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1351/2699] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:44 2022] PUT /identity/v3/projects/35023f0e4a0842778e3125cb024562c3/users/f322d73b5c5d41d69e197a767d7aea75/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:44.929396 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.929683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.929956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.933834 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0806e-c10e-4364-8a3b-be843a7099e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:44.936362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0806e-c10e-4364-8a3b-be843a7099e2 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:44.947794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1349/2700] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:44.949262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9a92-5873-413f-98bb-ee5b9e3d022d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:44.949689 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9a92-5873-413f-98bb-ee5b9e3d022d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:44.950062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9a92-5873-413f-98bb-ee5b9e3d022d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:44.951368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:44.952593 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:44.968428 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-e4da9a92-5873-413f-98bb-ee5b9e3d022d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:44.969699 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e4da9a92-5873-413f-98bb-ee5b9e3d022d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.022114 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1352/2701] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:45.023328 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.025350 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.033158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1350/2702] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:45.034629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9a92-5873-413f-98bb-ee5b9e3d022d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.038640 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f086e6-ed50-478a-932d-850a0b1428f4 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.043992 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.045168 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.045450 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.045724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] PATH_INFO: `/v3/users/7489dcefd5e7476c957a00c7e980d0f1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.047055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7489dcefd5e7476c957a00c7e980d0f1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.047309 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9a92-5873-413f-98bb-ee5b9e3d022d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.049037 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.052396 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.053132 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.053384 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.053656 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.054399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:45.054691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.056721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.070024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1351/2703] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:45.071041 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.076346 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.109896 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7489dcefd5e7476c957a00c7e980d0f1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:45.123552 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7489dcefd5e7476c957a00c7e980d0f1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:45.124544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7489dcefd5e7476c957a00c7e980d0f1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:45.125254 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7489dcefd5e7476c957a00c7e980d0f1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:45.134119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7489dcefd5e7476c957a00c7e980d0f1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:45.150895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69330347-05c6-4b56-a479-152cd190e32f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.151865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1353/2704] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 14:59:45 2022] DELETE /identity/v3/users/7489dcefd5e7476c957a00c7e980d0f1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:45.153017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.155091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.210872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.211762 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.212160 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.212441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.215495 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.252055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b28ab6-b78a-4921-b14f-afce7b504f97 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.263486 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab16e44-67d0-44ec-9ba3-6a78d89f269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.264704 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab16e44-67d0-44ec-9ba3-6a78d89f269e admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.265017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab16e44-67d0-44ec-9ba3-6a78d89f269e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.265495 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab16e44-67d0-44ec-9ba3-6a78d89f269e admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.270417 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab16e44-67d0-44ec-9ba3-6a78d89f269e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.271653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab16e44-67d0-44ec-9ba3-6a78d89f269e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.301951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.303417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.310503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1354/2705] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:45 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:45.310503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.310503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.310503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.320980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1352/2706] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:45.330735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.347778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac77c1-ab36-43f5-94b6-246caeb74779 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.353971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:45.356118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.433047 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1355/2707] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:45.445472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.447025 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.449140 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.450193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.450803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.451236 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.452640 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.456361 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.507985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1353/2708] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:45 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:45.509760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.526256 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836340b9-280b-47c2-996a-8c7daee3a7ae None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.527296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87459-a73a-404c-bb1c-90cee0256611 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.531068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.531786 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.532151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.532560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.534858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.535669 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.536040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.536953 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.537702 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.540314 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.548646 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.550408 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.555688 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1354/2709] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:45.559895 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.565004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1356/2710] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 14:59:45.571841 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5f33e-cc3b-4bd4-ac7d-764c0ca0b7e7 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.578280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b53cb-8ab6-4908-a438-603f8be1c54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.579241 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b53cb-8ab6-4908-a438-603f8be1c54c admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.579902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b53cb-8ab6-4908-a438-603f8be1c54c admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.580188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b53cb-8ab6-4908-a438-603f8be1c54c admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.581002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b53cb-8ab6-4908-a438-603f8be1c54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.583603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b53cb-8ab6-4908-a438-603f8be1c54c admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.601001 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1355/2711] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:45.603995 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.624264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d1d8d-8e1d-456e-9a08-f9dc523aa094 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.634081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.635987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.637014 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.637198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] PATH_INFO: `/v3/projects/839dcd1fc851418bbf741f318f64f9db/users/eed9038c0fb749b68b100d70f482c2e1/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.666914 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=839dcd1fc851418bbf741f318f64f9db, user_id=eed9038c0fb749b68b100d70f482c2e1, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.670055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:45.696898 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1357/2712] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:45 2022] PUT /identity/v3/projects/839dcd1fc851418bbf741f318f64f9db/users/eed9038c0fb749b68b100d70f482c2e1/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:45.699514 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.700177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.700658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.754091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] MFA Rules not processed for user `eed9038c0fb749b68b100d70f482c2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:45.757046 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.880935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1356/2713] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:45.892848 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.895862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.937416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:45.938796 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:45.981262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706676d5-b77f-4658-afce-3e85c61212fd admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:45.985954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cc8dd-9756-43e1-b863-6a761b7953cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:45.986723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cc8dd-9756-43e1-b863-6a761b7953cf service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:45.987053 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cc8dd-9756-43e1-b863-6a761b7953cf service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:45.987377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cc8dd-9756-43e1-b863-6a761b7953cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:45.998240 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cc8dd-9756-43e1-b863-6a761b7953cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:45.999710 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cc8dd-9756-43e1-b863-6a761b7953cf service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.010393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1358/2714] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:46.065082 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0831ff-2c44-478c-9d51-b9df08b168f2 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:46.071423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:46.071423 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.072067 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.072497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] PATH_INFO: `/v3/projects/6238a65284054cc0b1524bd9645b0912` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.074178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6238a65284054cc0b1524bd9645b0912)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:46.076737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.130527 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6238a65284054cc0b1524bd9645b0912', 'request_id': 'req-a1784523-638c-4259-82e6-22b0d88d18bd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:46.139811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6238a65284054cc0b1524bd9645b0912', 'request_id': 'req-a1784523-638c-4259-82e6-22b0d88d18bd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:46.143216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6238a65284054cc0b1524bd9645b0912', 'request_id': 'req-a1784523-638c-4259-82e6-22b0d88d18bd'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:46.156562 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] The token cache is being invalidate because project 6238a65284054cc0b1524bd9645b0912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 14:59:46.156886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:46.157607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-a1784523-638c-4259-82e6-22b0d88d18bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 14:59:46.160835 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1357/2715] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 14:59:45 2022] DELETE /identity/v3/projects/6238a65284054cc0b1524bd9645b0912 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:46.166411 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f263-1e37-47e5-b3d9-8bb13663fcb0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.166749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f263-1e37-47e5-b3d9-8bb13663fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.167443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143f263-1e37-47e5-b3d9-8bb13663fcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.188393 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-a143f263-1e37-47e5-b3d9-8bb13663fcb0 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:46.189991 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a143f263-1e37-47e5-b3d9-8bb13663fcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:46.274198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1359/2716] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:46.282687 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513e9ce-8e0d-46f7-893e-d6f60b8922d5 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.283014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513e9ce-8e0d-46f7-893e-d6f60b8922d5 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.283524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513e9ce-8e0d-46f7-893e-d6f60b8922d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.321390 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-2513e9ce-8e0d-46f7-893e-d6f60b8922d5 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:46.323180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2513e9ce-8e0d-46f7-893e-d6f60b8922d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:46.337994 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1358/2717] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:46.339569 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143f263-1e37-47e5-b3d9-8bb13663fcb0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:46.355154 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143f263-1e37-47e5-b3d9-8bb13663fcb0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:46.362239 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:46.363177 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.363963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.364520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.365635 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 14:59:46.366171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:46.368108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.390329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513e9ce-8e0d-46f7-893e-d6f60b8922d5 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:46.397082 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1360/2718] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 14:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 14:59:46.401954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513e9ce-8e0d-46f7-893e-d6f60b8922d5 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:46.409078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:46.410934 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.411482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.411981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.413096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:46.416178 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.467244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1359/2719] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:46 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:46.470674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:46.487364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70a9a9-4761-4e02-8381-2aae0797ff8d admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:46.588690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:46.590331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.590462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.590921 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.592207 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:46.595559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.624625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1361/2720] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:46 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:46.630288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:46.653970 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee714-7049-4149-a533-ba8e3c498c1b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:46.663320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:46.665059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.665366 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.665831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.667182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:46.670948 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.692722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1360/2721] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 14:59:46.695295 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:46.707355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbef6-b032-40cb-a28a-238affd4da04 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:46.712924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8893e97-2cba-4a6c-bcf3-7773f57daac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:46.713612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893e97-2cba-4a6c-bcf3-7773f57daac4 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.714338 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893e97-2cba-4a6c-bcf3-7773f57daac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.714620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8893e97-2cba-4a6c-bcf3-7773f57daac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.715477 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8893e97-2cba-4a6c-bcf3-7773f57daac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:46.717905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8893e97-2cba-4a6c-bcf3-7773f57daac4 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.731218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1362/2722] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:46.736128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:46.750847 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86f20d-7cfa-44fe-ac45-7c27b9da9652 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:46.760358 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:46.762392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.762844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.763636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] PATH_INFO: `/v3/projects/0a60c17ae45844c490bcefc6bc54c879/users/bc3501400c3b40f7993871f59e8571ff/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.790352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a60c17ae45844c490bcefc6bc54c879, user_id=bc3501400c3b40f7993871f59e8571ff, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:46.793561 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:46.821198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efef82-4f28-46a5-a3c1-e87efff607fe None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:46.821493 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efef82-4f28-46a5-a3c1-e87efff607fe None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:46.821776 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efef82-4f28-46a5-a3c1-e87efff607fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:46.823568 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1361/2723] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:46 2022] PUT /identity/v3/projects/0a60c17ae45844c490bcefc6bc54c879/users/bc3501400c3b40f7993871f59e8571ff/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:46.848445 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-03efef82-4f28-46a5-a3c1-e87efff607fe None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:46.849784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-03efef82-4f28-46a5-a3c1-e87efff607fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:46.969191 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1363/2724] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:46.980992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:46.983764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:47.067790 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b515525-be47-46e3-a1c9-e93f64a9726b admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:47.071973 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:47.072694 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:47.073011 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:47.073286 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:47.088468 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:47.092983 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:47.105436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1362/2725] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:48.015854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bea32-0dbf-44d7-a8bb-8c57e96aad6e None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:48.016928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bea32-0dbf-44d7-a8bb-8c57e96aad6e None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:48.017277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13bea32-0dbf-44d7-a8bb-8c57e96aad6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:48.065322 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f13bea32-0dbf-44d7-a8bb-8c57e96aad6e None None] MFA Rules not processed for user `bc3501400c3b40f7993871f59e8571ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:48.067816 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f13bea32-0dbf-44d7-a8bb-8c57e96aad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:48.277651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1364/2726] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:48.289781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:48.292587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:48.379242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3999c-fdb4-4a0a-8345-eadf0258acff service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:48.386238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac96b7-98c9-40ca-81df-bd1a1e72b243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:48.387201 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac96b7-98c9-40ca-81df-bd1a1e72b243 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:48.387710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac96b7-98c9-40ca-81df-bd1a1e72b243 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:48.388093 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac96b7-98c9-40ca-81df-bd1a1e72b243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:48.398849 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac96b7-98c9-40ca-81df-bd1a1e72b243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:48.402049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac96b7-98c9-40ca-81df-bd1a1e72b243 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:48.420032 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1363/2727] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 14:59:50.069729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:50.069729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:50.070674 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:50.101559 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] MFA Rules not processed for user `bc3501400c3b40f7993871f59e8571ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:50.104682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:50.122509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1365/2728] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:51.229002 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6398ed-4b39-41cc-a607-0d6ea6fa04e9 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:51.229452 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6398ed-4b39-41cc-a607-0d6ea6fa04e9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:51.229857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6398ed-4b39-41cc-a607-0d6ea6fa04e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:51.380441 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-aa6398ed-4b39-41cc-a607-0d6ea6fa04e9 None None] MFA Rules not processed for user `f322d73b5c5d41d69e197a767d7aea75`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:51.383037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-aa6398ed-4b39-41cc-a607-0d6ea6fa04e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:51.533873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1364/2729] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:51.537507 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:51.539659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:51.569869 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f32ca-241d-414e-8c4f-bf46baa80a95 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:51.578754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:51.580027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:51.580993 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:51.581965 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:51.583503 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:51.587508 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:51.629323 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1366/2730] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 14:59:51 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 14:59:51.634383 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6398ed-4b39-41cc-a607-0d6ea6fa04e9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:51.646979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6398ed-4b39-41cc-a607-0d6ea6fa04e9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:51.735877 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:51.736618 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:51.737230 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:51.737754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:51.738531 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:51.740646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:51.779611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1365/2731] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 14:59:51 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 14:59:51.781936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:51.795682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de58f3-0d16-443d-9ee6-226244a6c47f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:51.808866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:51.809579 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:51.809866 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:51.810164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:51.810944 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:51.814766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:51.851851 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1367/2732] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:51.853555 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:51.872121 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a007f6-57de-4ca4-88a9-cacc65bc0560 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:51.877460 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:51.878372 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:51.878653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:51.878923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] PATH_INFO: `/v3/projects/b7a7cbefee9640cda9550c3dff2991ac/users/36e88925b2674bdfa2bbfe8b04bd759a/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:51.915692 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a7cbefee9640cda9550c3dff2991ac, user_id=36e88925b2674bdfa2bbfe8b04bd759a, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:51.922097 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:51.974029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1366/2733] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:51 2022] PUT /identity/v3/projects/b7a7cbefee9640cda9550c3dff2991ac/users/36e88925b2674bdfa2bbfe8b04bd759a/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 14:59:51.978517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:52.000474 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34de64-9e88-474e-95b5-878d10b7838f admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:52.164320 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae494287-c899-4c42-b362-c31334ba1b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:52.166527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae494287-c899-4c42-b362-c31334ba1b83 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:52.168358 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae494287-c899-4c42-b362-c31334ba1b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:52.168939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae494287-c899-4c42-b362-c31334ba1b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:52.170148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae494287-c899-4c42-b362-c31334ba1b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:52.172377 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae494287-c899-4c42-b362-c31334ba1b83 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:52.189220 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1368/2734] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 14:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 14:59:52.191402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:52.208040 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d052a-e118-49f1-b87b-6ba3b6000e76 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:52.213921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:52.215191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:52.215631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:52.216048 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] PATH_INFO: `/v3/projects/b7a7cbefee9640cda9550c3dff2991ac/users/36e88925b2674bdfa2bbfe8b04bd759a/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:52.218490 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a7cbefee9640cda9550c3dff2991ac, user_id=36e88925b2674bdfa2bbfe8b04bd759a, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:52.221538 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:52.240506 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1367/2735] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 14:59:52 2022] PUT /identity/v3/projects/b7a7cbefee9640cda9550c3dff2991ac/users/36e88925b2674bdfa2bbfe8b04bd759a/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 14:59:58.252174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea45f3-5efc-432a-83b7-08138ec3da69 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:58.252823 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea45f3-5efc-432a-83b7-08138ec3da69 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:58.253224 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea45f3-5efc-432a-83b7-08138ec3da69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:58.293742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6cea45f3-5efc-432a-83b7-08138ec3da69 None None] MFA Rules not processed for user `36e88925b2674bdfa2bbfe8b04bd759a`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 14:59:58.295904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6cea45f3-5efc-432a-83b7-08138ec3da69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:58.422085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1369/2736] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 14:59:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 14:59:58.434535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 14:59:58.437661 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 14:59:58.545957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b57d52-88ba-456a-ae0f-533b4c18bc9f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 14:59:58.556272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cae9c-0481-4075-8689-9a784c2ab6a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 14:59:58.557487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cae9c-0481-4075-8689-9a784c2ab6a7 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 14:59:58.557996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cae9c-0481-4075-8689-9a784c2ab6a7 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 14:59:58.558598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cae9c-0481-4075-8689-9a784c2ab6a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 14:59:58.659758 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cae9c-0481-4075-8689-9a784c2ab6a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 14:59:58.661751 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3cae9c-0481-4075-8689-9a784c2ab6a7 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 14:59:58.674227 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1368/2737] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:00.571984 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:00.572523 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:00.572661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:00.591020 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] MFA Rules not processed for user `f322d73b5c5d41d69e197a767d7aea75`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:00.592245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:00.599697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1370/2738] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:00:19.573426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:19.574065 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:19.574345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:19.596232 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] MFA Rules not processed for user `3075413bbcba4104bb5fbb907ab2558a`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:19.597497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:19.662593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1369/2739] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:20.532878 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:20.534502 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:20.629591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3b363-c64a-41f6-9a03-f730b74c309b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:20.634774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:20.635730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:20.636009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:20.636282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] PATH_INFO: `/v3/users/c639c81dc43d427aa0e9962ad1fb6f44` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:20.637619 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c639c81dc43d427aa0e9962ad1fb6f44)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:20.639825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:20.681377 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:20.685012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:20.703749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c639c81dc43d427aa0e9962ad1fb6f44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:20.716628 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c639c81dc43d427aa0e9962ad1fb6f44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:20.717542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c639c81dc43d427aa0e9962ad1fb6f44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:20.718215 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c639c81dc43d427aa0e9962ad1fb6f44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:20.726512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-19ec197e-0a4f-4a05-88d4-e1a67b0ea76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c639c81dc43d427aa0e9962ad1fb6f44'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:20.739912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1371/2740] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:00:20 2022] DELETE /identity/v3/users/c639c81dc43d427aa0e9962ad1fb6f44 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 15:00:20.746976 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:20.747319 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:20.747594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:20.761703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0e057-9151-43a9-b154-a22bd85c7ac0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:20.774808 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:20.776125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:20.837587 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:20.838328 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:20.838642 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:20.838927 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:20.841668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:20.854815 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1372/2741] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:20.866702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:20.868742 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:20.914085 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:20.915924 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:20.926198 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1370/2742] 10.222.0.32 () {60 vars in 1396 bytes} [Wed May 4 15:00:20 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3246 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 15:00:20.929068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf703be-ddf9-4746-b8a7-9a9e16d8f1c6 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:20.932999 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:20.933658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:20.933941 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:20.934217 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:20.943895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:20.945385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:20.955956 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1373/2743] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:21.318050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:21.319542 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:21.336889 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bbd54-616a-402f-b7f5-ad37a337d6f4 service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:21.348028 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:21.351022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:21.351623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:21.352116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] PATH_INFO: `/v3/projects/57078fd144594c0d82ae638a8ea5a631` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:21.356123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57078fd144594c0d82ae638a8ea5a631)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:21.358946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:21.422787 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57078fd144594c0d82ae638a8ea5a631', 'request_id': 'req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.431182 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57078fd144594c0d82ae638a8ea5a631', 'request_id': 'req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.437857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57078fd144594c0d82ae638a8ea5a631', 'request_id': 'req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.455625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] The token cache is being invalidate because project 57078fd144594c0d82ae638a8ea5a631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:00:21.456050 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.456910 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.460828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1371/2744] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:00:21 2022] DELETE /identity/v3/projects/57078fd144594c0d82ae638a8ea5a631 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:21.462591 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:21.478310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6cc7-01ed-43b1-86d7-39b9fd14afc3 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:21.572599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:21.573604 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:21.573895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:21.574171 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] PATH_INFO: `/v3/users/9084adee55b048f68f24931b8adeeb15` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:21.575457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9084adee55b048f68f24931b8adeeb15)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:21.577453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:21.641479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9084adee55b048f68f24931b8adeeb15'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.658542 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9084adee55b048f68f24931b8adeeb15'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.659653 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9084adee55b048f68f24931b8adeeb15'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.660622 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9084adee55b048f68f24931b8adeeb15'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.677958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9084adee55b048f68f24931b8adeeb15'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:21.698925 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1374/2745] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:00:21 2022] DELETE /identity/v3/users/9084adee55b048f68f24931b8adeeb15 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:21.774873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:21.777306 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:21.902588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc54b3-bb7b-4d5c-8d7b-0c02bcf019a0 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:21.980351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33190543-9886-4558-b4b6-a55473f07e40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac21bca8da642338571f821ef5ea121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:21.981051 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33190543-9886-4558-b4b6-a55473f07e40 service placement] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:21.981326 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33190543-9886-4558-b4b6-a55473f07e40 service placement] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:21.981596 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33190543-9886-4558-b4b6-a55473f07e40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:21.983671 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-33190543-9886-4558-b4b6-a55473f07e40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:22.010187 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33190543-9886-4558-b4b6-a55473f07e40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.011873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33190543-9886-4558-b4b6-a55473f07e40 service placement] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.028714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1372/2746] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:22.067244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:22.068726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:22.150699 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac72c11-34ad-4fd2-8a80-eb03b0aa0460 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:22.156666 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:22.157682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.157928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.158204 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] PATH_INFO: `/v3/projects/1d26724a702b461a982ea5af96bed781` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.159443 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d26724a702b461a982ea5af96bed781)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.162394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.224102 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d26724a702b461a982ea5af96bed781', 'request_id': 'req-6de72567-fa37-45b6-842d-8b0933bbd24a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:22.235148 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d26724a702b461a982ea5af96bed781', 'request_id': 'req-6de72567-fa37-45b6-842d-8b0933bbd24a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:22.241650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d26724a702b461a982ea5af96bed781', 'request_id': 'req-6de72567-fa37-45b6-842d-8b0933bbd24a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:22.260242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] The token cache is being invalidate because project 1d26724a702b461a982ea5af96bed781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:00:22.260678 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:22.261764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-6de72567-fa37-45b6-842d-8b0933bbd24a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:22.263143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1375/2747] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:00:22 2022] DELETE /identity/v3/projects/1d26724a702b461a982ea5af96bed781 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:22.269793 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6b5b-cc69-4fd6-8a55-0a2c5a212b7d None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.270084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6b5b-cc69-4fd6-8a55-0a2c5a212b7d None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.270343 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e6b5b-cc69-4fd6-8a55-0a2c5a212b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.289980 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-207e6b5b-cc69-4fd6-8a55-0a2c5a212b7d None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:22.291386 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-207e6b5b-cc69-4fd6-8a55-0a2c5a212b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:22.382427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1373/2748] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:22.390651 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82861-7c1b-4e3e-b152-1eb6d53fd7dd None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.390970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82861-7c1b-4e3e-b152-1eb6d53fd7dd None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.391208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82861-7c1b-4e3e-b152-1eb6d53fd7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.411405 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-94d82861-7c1b-4e3e-b152-1eb6d53fd7dd None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:22.412625 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-94d82861-7c1b-4e3e-b152-1eb6d53fd7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:22.423879 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1376/2749] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:00:22.429394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e6b5b-cc69-4fd6-8a55-0a2c5a212b7d None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:22.446516 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e6b5b-cc69-4fd6-8a55-0a2c5a212b7d None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:22.454021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:22.455331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.455771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.456211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.457681 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 15:00:22.458255 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.460982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.481727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1374/2750] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 15:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 15:00:22.489970 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82861-7c1b-4e3e-b152-1eb6d53fd7dd None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:22.506023 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82861-7c1b-4e3e-b152-1eb6d53fd7dd None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:22.514236 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:22.515259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.515735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.516214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.517550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.521155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.561352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1377/2751] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:00:22 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:22.566030 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:22.577128 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51fba5-f467-4ffe-932d-c2b9c15332f7 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:22.654901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:22.655407 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.655724 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.656016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.656771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.658803 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.681096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1375/2752] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:00:22 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:22.685811 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:22.703716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd8e6e-6b24-4a17-8033-2266863060b4 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:22.708987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:22.709675 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.709953 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.710219 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.710960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.712656 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.726512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1378/2753] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:00:22.730552 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:22.742857 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aa3f2-f856-448e-a53c-b3356c341d39 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:22.750138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c26fe-a9d5-4144-92e0-3fc8b7c3bc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:22.750770 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c26fe-a9d5-4144-92e0-3fc8b7c3bc2f admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.751149 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c26fe-a9d5-4144-92e0-3fc8b7c3bc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.751512 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c26fe-a9d5-4144-92e0-3fc8b7c3bc2f admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.752354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c26fe-a9d5-4144-92e0-3fc8b7c3bc2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.754448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c26fe-a9d5-4144-92e0-3fc8b7c3bc2f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.765505 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1376/2754] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 15:00:22.770513 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:22.784403 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9e6c-a2ec-49c8-8a02-8644aff2b626 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:22.796567 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:22.798018 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.798390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.798733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] PATH_INFO: `/v3/projects/9de6cf6271ec43e7ab3b2018b60e3178/users/ba94617281cb41398d95a7c403c2dfd1/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.826736 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de6cf6271ec43e7ab3b2018b60e3178, user_id=ba94617281cb41398d95a7c403c2dfd1, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:22.830357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:22.858044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1379/2755] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:00:22 2022] PUT /identity/v3/projects/9de6cf6271ec43e7ab3b2018b60e3178/users/ba94617281cb41398d95a7c403c2dfd1/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:22.871627 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6506b39-fd6a-4cf5-a084-1377e83de7bd None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:22.872245 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6506b39-fd6a-4cf5-a084-1377e83de7bd None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:22.874829 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6506b39-fd6a-4cf5-a084-1377e83de7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:22.917691 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-b6506b39-fd6a-4cf5-a084-1377e83de7bd None None] MFA Rules not processed for user `ba94617281cb41398d95a7c403c2dfd1`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:22.919216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-b6506b39-fd6a-4cf5-a084-1377e83de7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:22.999817 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1377/2756] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:23.004880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:23.085140 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff530fb-af1f-4bf3-9e15-07c9ae6f4269 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:23.091010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:23.092339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.092339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.092588 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.093163 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:23.095226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:23.132623 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1380/2757] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:00:23 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 15:00:23.137603 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6506b39-fd6a-4cf5-a084-1377e83de7bd None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:23.156978 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6506b39-fd6a-4cf5-a084-1377e83de7bd None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:23.248288 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:23.248902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.249189 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.249466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.250192 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:23.252436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:23.277367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1378/2758] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:00:23 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:23.281959 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:23.297448 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5cf32-3fa5-4ae1-8d22-b17235d65907 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:23.304524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:23.305528 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.305865 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.306932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.307721 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:23.309439 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:23.323022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1381/2759] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:00:23.327216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:23.341741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e4a6b-d757-43f4-8a8c-93e18b597a29 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:23.349111 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:23.350107 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.350466 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.350744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] PATH_INFO: `/v3/projects/dd6dc69a098243bb934b90698bff243b/users/d57fa83091f943b5a12813d56031bf42/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.375875 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6dc69a098243bb934b90698bff243b, user_id=d57fa83091f943b5a12813d56031bf42, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:23.382260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:23.406014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1379/2760] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:00:23 2022] PUT /identity/v3/projects/dd6dc69a098243bb934b90698bff243b/users/d57fa83091f943b5a12813d56031bf42/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:23.410942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:23.429310 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447fb0-3e06-455d-acc8-af66cceb3c83 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:23.527181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b098c11-f0e4-4d6a-9092-6718fcd2dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:23.527779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b098c11-f0e4-4d6a-9092-6718fcd2dce1 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.528062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b098c11-f0e4-4d6a-9092-6718fcd2dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.528341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b098c11-f0e4-4d6a-9092-6718fcd2dce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.529119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b098c11-f0e4-4d6a-9092-6718fcd2dce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:23.530734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b098c11-f0e4-4d6a-9092-6718fcd2dce1 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:23.541872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1382/2761] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 15:00:23.546338 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:23.562923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d7da3-aec1-478f-b34b-070d6265f3db admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:23.569554 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:23.570490 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.570798 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.571205 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] PATH_INFO: `/v3/projects/dd6dc69a098243bb934b90698bff243b/users/d57fa83091f943b5a12813d56031bf42/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.574981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6dc69a098243bb934b90698bff243b, user_id=d57fa83091f943b5a12813d56031bf42, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:23.578872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:23.591330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1380/2762] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:00:23 2022] PUT /identity/v3/projects/dd6dc69a098243bb934b90698bff243b/users/d57fa83091f943b5a12813d56031bf42/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:23.608841 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe39f-dc95-4467-a7f6-36f45ab3b7b1 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.609412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe39f-dc95-4467-a7f6-36f45ab3b7b1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.610174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fe39f-dc95-4467-a7f6-36f45ab3b7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.671147 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-645fe39f-dc95-4467-a7f6-36f45ab3b7b1 None None] MFA Rules not processed for user `d57fa83091f943b5a12813d56031bf42`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:23.673544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-645fe39f-dc95-4467-a7f6-36f45ab3b7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:23.771394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1383/2763] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:00:23.784869 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:23.787014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:23.882417 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09754548-f5d4-45ec-a5a6-45b907f5fa6a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:23.890206 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdccdd2-4602-449d-baf4-433c9f6a6c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:23.890986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdccdd2-4602-449d-baf4-433c9f6a6c17 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:23.891335 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdccdd2-4602-449d-baf4-433c9f6a6c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:23.891682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdccdd2-4602-449d-baf4-433c9f6a6c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:23.981293 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdccdd2-4602-449d-baf4-433c9f6a6c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:23.984485 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdccdd2-4602-449d-baf4-433c9f6a6c17 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:24.002063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1381/2764] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:26.737081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fe39f-dc95-4467-a7f6-36f45ab3b7b1 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:26.807104 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fe39f-dc95-4467-a7f6-36f45ab3b7b1 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:26.813620 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:26.814805 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:26.815262 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:26.815731 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] PATH_INFO: `/v3/users/ba94617281cb41398d95a7c403c2dfd1` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:26.817867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba94617281cb41398d95a7c403c2dfd1)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:26.821683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:26.891195 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba94617281cb41398d95a7c403c2dfd1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:26.907080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba94617281cb41398d95a7c403c2dfd1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:26.907971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba94617281cb41398d95a7c403c2dfd1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:26.908982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba94617281cb41398d95a7c403c2dfd1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:26.918506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba94617281cb41398d95a7c403c2dfd1'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:26.936108 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1384/2765] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:00:26 2022] DELETE /identity/v3/users/ba94617281cb41398d95a7c403c2dfd1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:26.941197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:26.942079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:26.942079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:26.960307 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:26.961658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:27.035865 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1382/2766] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:27.046727 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:27.053854 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:27.122828 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a506a-87f3-45ec-92dc-9741cc11d64b admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:27.127181 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:27.128246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:27.128694 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:27.129599 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:27.141202 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:27.142938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:27.154572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1385/2767] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:27.521738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:27.524049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:27.543076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa14004-bd92-4b15-b345-f48c58c51072 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:27.551049 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:27.552069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:27.552355 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:27.552750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] PATH_INFO: `/v3/projects/9de6cf6271ec43e7ab3b2018b60e3178` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:27.554367 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de6cf6271ec43e7ab3b2018b60e3178)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:27.556615 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:27.617489 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de6cf6271ec43e7ab3b2018b60e3178', 'request_id': 'req-14cacf61-a6b9-4800-bb08-a5758040003d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.627106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de6cf6271ec43e7ab3b2018b60e3178', 'request_id': 'req-14cacf61-a6b9-4800-bb08-a5758040003d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.635052 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de6cf6271ec43e7ab3b2018b60e3178', 'request_id': 'req-14cacf61-a6b9-4800-bb08-a5758040003d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.648921 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] The token cache is being invalidate because project 9de6cf6271ec43e7ab3b2018b60e3178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:00:27.649228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.649861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.651484 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1383/2768] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:00:27 2022] DELETE /identity/v3/projects/9de6cf6271ec43e7ab3b2018b60e3178 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:27.655762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:27.668438 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884d8ca-4c7d-424a-829e-7a23f76b271b service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:27.727391 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:27.728218 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:27.728488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:27.728764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] PATH_INFO: `/v3/users/d57fa83091f943b5a12813d56031bf42` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:27.731035 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d57fa83091f943b5a12813d56031bf42)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:27.733088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:27.791091 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd57fa83091f943b5a12813d56031bf42'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.805321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd57fa83091f943b5a12813d56031bf42'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.806764 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd57fa83091f943b5a12813d56031bf42'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.807422 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd57fa83091f943b5a12813d56031bf42'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.816520 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-24249224-b47b-44bd-ba98-27e6a8d74dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd57fa83091f943b5a12813d56031bf42'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:27.829807 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1386/2769] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:00:27 2022] DELETE /identity/v3/users/d57fa83091f943b5a12813d56031bf42 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:28.140352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.142340 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:28.158731 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cacf61-a6b9-4800-bb08-a5758040003d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:28.226235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:28.227155 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.227440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.227718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] PATH_INFO: `/v3/projects/dd6dc69a098243bb934b90698bff243b` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.228999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6dc69a098243bb934b90698bff243b)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:28.231122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:28.286108 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6dc69a098243bb934b90698bff243b', 'request_id': 'req-88589a92-116c-4b4f-83e0-0d8ab0140434'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:28.293617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6dc69a098243bb934b90698bff243b', 'request_id': 'req-88589a92-116c-4b4f-83e0-0d8ab0140434'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:28.301084 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6dc69a098243bb934b90698bff243b', 'request_id': 'req-88589a92-116c-4b4f-83e0-0d8ab0140434'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:28.321631 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] The token cache is being invalidate because project dd6dc69a098243bb934b90698bff243b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:00:28.322436 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:28.324317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-88589a92-116c-4b4f-83e0-0d8ab0140434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:28.332534 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1384/2770] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:00:28 2022] DELETE /identity/v3/projects/dd6dc69a098243bb934b90698bff243b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:28.334769 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ab799-b5f0-46ee-ac56-8f39417a7c05 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.335062 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ab799-b5f0-46ee-ac56-8f39417a7c05 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.335339 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ab799-b5f0-46ee-ac56-8f39417a7c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.354990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-8c0ab799-b5f0-46ee-ac56-8f39417a7c05 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:28.356234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8c0ab799-b5f0-46ee-ac56-8f39417a7c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:28.430418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1387/2771] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:28.440946 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9218e6d-0b63-4e3a-9d95-ce970834bbf9 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.441350 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9218e6d-0b63-4e3a-9d95-ce970834bbf9 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.441753 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9218e6d-0b63-4e3a-9d95-ce970834bbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.463153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-a9218e6d-0b63-4e3a-9d95-ce970834bbf9 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:28.464458 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-a9218e6d-0b63-4e3a-9d95-ce970834bbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:28.472902 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1385/2772] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:00:28.477500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ab799-b5f0-46ee-ac56-8f39417a7c05 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.487728 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ab799-b5f0-46ee-ac56-8f39417a7c05 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:28.493066 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:28.493583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.493867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.494142 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.494932 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 15:00:28.495242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:28.496935 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:28.513225 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1388/2773] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 15:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 15:00:28.516641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9218e6d-0b63-4e3a-9d95-ce970834bbf9 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.526872 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9218e6d-0b63-4e3a-9d95-ce970834bbf9 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:28.532427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:28.533300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.533646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.533984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.534721 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:28.536881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:28.571511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1386/2774] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:00:28 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 15:00:28.576606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.590813 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2f9ed-5a0e-4b42-b837-8ad1e4b50da0 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:28.671401 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:28.672125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.672413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.672729 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.673465 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:28.675794 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:28.698818 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1389/2775] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:00:28 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:28.703507 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.715999 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30027d-ea2a-47b3-8d05-96e33d3e0024 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:28.721682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:28.722411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.722695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.722972 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.723754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:28.725477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:28.738402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1387/2776] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:00:28.742990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.754928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98b4e-eba0-4ea6-91c5-7638f4046c85 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:28.761734 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88782b-f137-41f3-8ec3-85c420664e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:28.762475 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88782b-f137-41f3-8ec3-85c420664e9e admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.762796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88782b-f137-41f3-8ec3-85c420664e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.763098 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88782b-f137-41f3-8ec3-85c420664e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.763904 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88782b-f137-41f3-8ec3-85c420664e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:28.765940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88782b-f137-41f3-8ec3-85c420664e9e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:28.779517 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1390/2777] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:00:28.784330 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.795469 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf6d3-4ed8-4040-a7a8-69ac00423995 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:28.800592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:28.801730 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.801919 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.802260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] PATH_INFO: `/v3/projects/127c998138334dacab6b66ac1ded5041/users/8c337589ef3f4761a61f9ed70c294c28/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.826068 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127c998138334dacab6b66ac1ded5041, user_id=8c337589ef3f4761a61f9ed70c294c28, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:28.828977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:28.845352 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1388/2778] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:00:28 2022] PUT /identity/v3/projects/127c998138334dacab6b66ac1ded5041/users/8c337589ef3f4761a61f9ed70c294c28/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 15:00:28.857214 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac93776-8a9d-4b29-98c5-f44cd9f86e9a None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:28.857820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac93776-8a9d-4b29-98c5-f44cd9f86e9a None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:28.857820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac93776-8a9d-4b29-98c5-f44cd9f86e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:28.891983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-0ac93776-8a9d-4b29-98c5-f44cd9f86e9a None None] MFA Rules not processed for user `8c337589ef3f4761a61f9ed70c294c28`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:28.893281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-0ac93776-8a9d-4b29-98c5-f44cd9f86e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:28.965459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1391/2779] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:28.979497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:28.981405 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:29.055860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aceea-4ee4-4a11-b692-dc9911ab93c5 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:29.060194 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec192f-6323-42ec-b0a1-7b2fa1375924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:29.060954 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec192f-6323-42ec-b0a1-7b2fa1375924 service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:29.061163 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec192f-6323-42ec-b0a1-7b2fa1375924 service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:29.061439 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec192f-6323-42ec-b0a1-7b2fa1375924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:29.072242 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec192f-6323-42ec-b0a1-7b2fa1375924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:29.073992 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec192f-6323-42ec-b0a1-7b2fa1375924 service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:29.084448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1389/2780] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:31.694394 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac93776-8a9d-4b29-98c5-f44cd9f86e9a None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:31.779187 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac93776-8a9d-4b29-98c5-f44cd9f86e9a None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:31.786911 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:31.788485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:31.789042 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:31.789541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] PATH_INFO: `/v3/users/8c337589ef3f4761a61f9ed70c294c28` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:31.791711 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c337589ef3f4761a61f9ed70c294c28)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:31.795398 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:31.866647 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c337589ef3f4761a61f9ed70c294c28'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:31.882617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c337589ef3f4761a61f9ed70c294c28'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:31.883372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c337589ef3f4761a61f9ed70c294c28'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:31.883989 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c337589ef3f4761a61f9ed70c294c28'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:31.895021 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c337589ef3f4761a61f9ed70c294c28'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:31.916757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1392/2781] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:00:31 2022] DELETE /identity/v3/users/8c337589ef3f4761a61f9ed70c294c28 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:31.922501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:31.923035 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:31.923188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:31.945014 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:31.947120 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:32.037016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1390/2782] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:32.050833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:32.050833 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:32.137415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e145f0-e509-4348-9633-0033c1d16142 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:32.139230 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1b33c-cf57-4e66-b0c8-514b0c9b3bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:32.139946 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b33c-cf57-4e66-b0c8-514b0c9b3bdc service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:32.140455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b33c-cf57-4e66-b0c8-514b0c9b3bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:32.140741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1b33c-cf57-4e66-b0c8-514b0c9b3bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:32.151825 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1b33c-cf57-4e66-b0c8-514b0c9b3bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:32.153419 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1b33c-cf57-4e66-b0c8-514b0c9b3bdc service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:32.165840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1393/2783] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:32.489322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:32.491406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:32.536019 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61bad9-dc61-4030-b370-df9e47757509 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:32.545351 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:32.546636 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:32.547076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:32.547583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] PATH_INFO: `/v3/projects/127c998138334dacab6b66ac1ded5041` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:32.550750 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=127c998138334dacab6b66ac1ded5041)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:32.556776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:32.630602 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127c998138334dacab6b66ac1ded5041', 'request_id': 'req-082a73ca-74c5-4f8b-b578-9235a12251a2'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:32.638138 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127c998138334dacab6b66ac1ded5041', 'request_id': 'req-082a73ca-74c5-4f8b-b578-9235a12251a2'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:32.646831 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127c998138334dacab6b66ac1ded5041', 'request_id': 'req-082a73ca-74c5-4f8b-b578-9235a12251a2'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:32.667860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] The token cache is being invalidate because project 127c998138334dacab6b66ac1ded5041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:00:32.668173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:32.668894 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-082a73ca-74c5-4f8b-b578-9235a12251a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:32.671664 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1391/2784] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:00:32 2022] DELETE /identity/v3/projects/127c998138334dacab6b66ac1ded5041 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:32.677245 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aeed1-0077-40c9-a090-afd1008371bc None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:32.677627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aeed1-0077-40c9-a090-afd1008371bc None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:32.677998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aeed1-0077-40c9-a090-afd1008371bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:32.702880 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-d75aeed1-0077-40c9-a090-afd1008371bc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:32.705096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-d75aeed1-0077-40c9-a090-afd1008371bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:32.793650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1394/2785] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:32.810942 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6a0fe-54a0-44c9-b6c9-7b63064388bc None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:32.811299 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6a0fe-54a0-44c9-b6c9-7b63064388bc None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:32.813791 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6a0fe-54a0-44c9-b6c9-7b63064388bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:32.858275 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-24b6a0fe-54a0-44c9-b6c9-7b63064388bc None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:32.859937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-24b6a0fe-54a0-44c9-b6c9-7b63064388bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:32.876916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1392/2786] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:00:32.880589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aeed1-0077-40c9-a090-afd1008371bc None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:32.893990 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aeed1-0077-40c9-a090-afd1008371bc None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:32.901013 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:32.902726 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:32.903176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:32.903595 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:32.905682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 15:00:32.906905 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:32.910302 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:32.935572 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1395/2787] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 15:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 15:00:32.942168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6a0fe-54a0-44c9-b6c9-7b63064388bc None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:32.964334 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6a0fe-54a0-44c9-b6c9-7b63064388bc None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:32.974706 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:32.976280 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:32.977026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:32.977784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:32.979593 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:32.984312 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:33.045106 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1393/2788] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:00:32 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:33.052352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:33.071927 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c996a96-7308-41b3-a05b-b8413fe17720 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:33.210079 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:33.211056 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:33.226006 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:33.227022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:33.228971 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:33.235488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:33.283249 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1396/2789] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:00:33 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:33.288844 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:33.331257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59f83-1714-4002-bafc-8cf508e557aa admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:33.345285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:33.345285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:33.345285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:33.345285 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:33.345657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:33.345657 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:33.391654 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1394/2790] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:00:33.392701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:33.409015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e40a82-30c1-48d8-8061-80239eb9e9ef admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:33.431008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376c069-675c-45f9-bde5-04cc0e901493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:33.432489 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376c069-675c-45f9-bde5-04cc0e901493 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:33.435364 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376c069-675c-45f9-bde5-04cc0e901493 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:33.435709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376c069-675c-45f9-bde5-04cc0e901493 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:33.437618 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376c069-675c-45f9-bde5-04cc0e901493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:33.441397 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376c069-675c-45f9-bde5-04cc0e901493 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:33.464715 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1397/2791] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 15:00:33.469577 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:33.485287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2c36f-52bf-4d22-acec-ba881e223493 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:33.498443 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:33.499613 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:33.500083 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:33.501058 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] PATH_INFO: `/v3/projects/063b0fdaa49a4cb6a7db0defa0e18f8d/users/b2609760306b4f0ca07caaa8b3568a3d/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:33.519582 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=063b0fdaa49a4cb6a7db0defa0e18f8d, user_id=b2609760306b4f0ca07caaa8b3568a3d, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:33.522752 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:33.552959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1395/2792] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:00:33 2022] PUT /identity/v3/projects/063b0fdaa49a4cb6a7db0defa0e18f8d/users/b2609760306b4f0ca07caaa8b3568a3d/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:33.565934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9234c-ca1c-4cdf-a11e-45d05b2f8550 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:33.566266 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9234c-ca1c-4cdf-a11e-45d05b2f8550 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:33.566519 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9234c-ca1c-4cdf-a11e-45d05b2f8550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:33.600650 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-7ff9234c-ca1c-4cdf-a11e-45d05b2f8550 None None] MFA Rules not processed for user `b2609760306b4f0ca07caaa8b3568a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:33.602918 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7ff9234c-ca1c-4cdf-a11e-45d05b2f8550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:33.719221 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1398/2793] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:33.734597 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:33.737148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:33.810223 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a33b6-2ef2-43a3-9e7b-ec1daadc20bc admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:33.813990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:33.814658 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:33.814935 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:33.815216 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:33.826238 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:33.828478 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:33.840197 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1396/2794] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:35.316307 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b386b8-7698-4013-b103-f74520c4dbd1 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:35.321141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b386b8-7698-4013-b103-f74520c4dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:35.322170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b386b8-7698-4013-b103-f74520c4dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:35.361085 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-34b386b8-7698-4013-b103-f74520c4dbd1 None None] MFA Rules not processed for user `b2609760306b4f0ca07caaa8b3568a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:35.362442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-34b386b8-7698-4013-b103-f74520c4dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:35.371461 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1399/2795] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:38.013870 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:38.015369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:38.113969 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5af9-d4b7-45ff-8009-4ccfdd6311dc service cinder] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:38.119059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:38.120224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:38.120511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:38.120813 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] PATH_INFO: `/v3/users/eed9038c0fb749b68b100d70f482c2e1` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:38.122218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed9038c0fb749b68b100d70f482c2e1)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:38.124196 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:38.185408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed9038c0fb749b68b100d70f482c2e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:38.186153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed9038c0fb749b68b100d70f482c2e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:38.187000 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed9038c0fb749b68b100d70f482c2e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:38.198487 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed9038c0fb749b68b100d70f482c2e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:38.220251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed9038c0fb749b68b100d70f482c2e1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:38.237623 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1397/2796] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:00:38 2022] DELETE /identity/v3/users/eed9038c0fb749b68b100d70f482c2e1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:38.242548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:38.243560 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:38.243830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:38.263078 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:38.264292 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:38.343680 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1400/2797] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:38.355882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:38.357550 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:38.428125 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded5767-145b-4279-ba69-355a5a463b5f admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:38.431979 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0baac7-587c-4772-a08f-1644edae364b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:38.432695 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0baac7-587c-4772-a08f-1644edae364b service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:38.433021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0baac7-587c-4772-a08f-1644edae364b service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:38.433325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0baac7-587c-4772-a08f-1644edae364b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:38.443363 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0baac7-587c-4772-a08f-1644edae364b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:38.444823 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0baac7-587c-4772-a08f-1644edae364b service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:38.455354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1398/2798] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:38.824189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:38.828843 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:38.861076 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb2f8-506b-42c0-bc4e-4f5a7a8b1bef None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:38.883388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:38.885276 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:38.886174 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:38.886702 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] PATH_INFO: `/v3/projects/839dcd1fc851418bbf741f318f64f9db` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:38.889506 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] RBAC: Authorizing `identity:delete_project(project_id=839dcd1fc851418bbf741f318f64f9db)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:38.899124 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:38.991636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839dcd1fc851418bbf741f318f64f9db', 'request_id': 'req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:39.006926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839dcd1fc851418bbf741f318f64f9db', 'request_id': 'req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:39.019881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839dcd1fc851418bbf741f318f64f9db', 'request_id': 'req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:39.062349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] The token cache is being invalidate because project 839dcd1fc851418bbf741f318f64f9db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:00:39.062895 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:39.063862 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9a0bf8a-f787-4ee1-b4c8-d0bcfde6019c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:39.068967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1401/2799] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:00:38 2022] DELETE /identity/v3/projects/839dcd1fc851418bbf741f318f64f9db => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:39.071063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5f706-dfc7-4f63-bd7e-12d2fa5dfc90 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.071558 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5f706-dfc7-4f63-bd7e-12d2fa5dfc90 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.071794 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5f706-dfc7-4f63-bd7e-12d2fa5dfc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.092625 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-69b5f706-dfc7-4f63-bd7e-12d2fa5dfc90 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:39.094341 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-69b5f706-dfc7-4f63-bd7e-12d2fa5dfc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:39.193641 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1399/2800] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:39.203783 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552361-8ae0-46ec-98c4-814b72a88556 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.204071 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552361-8ae0-46ec-98c4-814b72a88556 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.204636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552361-8ae0-46ec-98c4-814b72a88556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.245565 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-03552361-8ae0-46ec-98c4-814b72a88556 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:39.247356 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-03552361-8ae0-46ec-98c4-814b72a88556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:39.258265 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1402/2801] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:39.263583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5f706-dfc7-4f63-bd7e-12d2fa5dfc90 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.275453 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5f706-dfc7-4f63-bd7e-12d2fa5dfc90 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.281027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:39.281824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.282168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.282662 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.283608 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 15:00:39.284158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:39.286007 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:39.303191 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1400/2802] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 15:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 15:00:39.313234 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03552361-8ae0-46ec-98c4-814b72a88556 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.330797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03552361-8ae0-46ec-98c4-814b72a88556 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.335697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:39.336434 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.336824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.337092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.337876 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:39.340008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:39.374026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1403/2803] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:00:39 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:39.379578 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.391195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b665d-3fbe-453f-ae9e-41861ba1c857 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.455907 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:39.456703 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.457105 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.457474 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.458524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:39.460735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:39.493025 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1401/2804] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:00:39 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:00:39.494594 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.506415 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0752d4-899d-46a6-8c65-761f710dbcd3 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.513275 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:39.514514 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.515106 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.515538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.516462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:39.518287 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:39.533568 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1404/2805] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:00:39.537819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.549317 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802620f-eb2f-4750-a731-1189321c9c26 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.556371 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f63dd-45a5-410b-8cc0-504c1b4026bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:39.557229 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f63dd-45a5-410b-8cc0-504c1b4026bc admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.557511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f63dd-45a5-410b-8cc0-504c1b4026bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.557861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f63dd-45a5-410b-8cc0-504c1b4026bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.558723 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f63dd-45a5-410b-8cc0-504c1b4026bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:39.560734 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f63dd-45a5-410b-8cc0-504c1b4026bc admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:39.574399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1402/2806] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:00:39.578652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.590115 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd32d2c-041d-4833-849a-3dfce0395e07 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.594868 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:39.595766 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.596048 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.596317 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] PATH_INFO: `/v3/projects/ec5ad031468148f8947a0cbdb453e2c5/users/af34317cec894fe4a96f4eec053f183a/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.617001 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5ad031468148f8947a0cbdb453e2c5, user_id=af34317cec894fe4a96f4eec053f183a, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:39.620203 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:39.636389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1405/2807] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:00:39 2022] PUT /identity/v3/projects/ec5ad031468148f8947a0cbdb453e2c5/users/af34317cec894fe4a96f4eec053f183a/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:39.654421 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc3d9-5707-42c8-97b1-4e10806997fb None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.654741 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc3d9-5707-42c8-97b1-4e10806997fb None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.655621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc3d9-5707-42c8-97b1-4e10806997fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.689244 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-5fbdc3d9-5707-42c8-97b1-4e10806997fb None None] MFA Rules not processed for user `af34317cec894fe4a96f4eec053f183a`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:39.690609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-5fbdc3d9-5707-42c8-97b1-4e10806997fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:39.770789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1403/2808] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:39.784826 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.787051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:39.852703 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd57920-eb36-4bc0-966d-d5392f32f198 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.856557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915ad7f-e1ed-4276-8ba5-7a7fe56dddfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:39.857255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915ad7f-e1ed-4276-8ba5-7a7fe56dddfa service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:39.857543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915ad7f-e1ed-4276-8ba5-7a7fe56dddfa service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:39.857822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915ad7f-e1ed-4276-8ba5-7a7fe56dddfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:39.867027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915ad7f-e1ed-4276-8ba5-7a7fe56dddfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:39.868500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915ad7f-e1ed-4276-8ba5-7a7fe56dddfa service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:39.878466 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1406/2809] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:39.910614 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc3d9-5707-42c8-97b1-4e10806997fb None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:39.993451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc3d9-5707-42c8-97b1-4e10806997fb None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:39.999959 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:40.001445 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:40.001832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:40.002184 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] PATH_INFO: `/v3/users/af34317cec894fe4a96f4eec053f183a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:40.004327 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af34317cec894fe4a96f4eec053f183a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:40.006415 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:40.088164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af34317cec894fe4a96f4eec053f183a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:40.089303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af34317cec894fe4a96f4eec053f183a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:40.090451 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af34317cec894fe4a96f4eec053f183a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:40.102744 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af34317cec894fe4a96f4eec053f183a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:40.120333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af34317cec894fe4a96f4eec053f183a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:40.137283 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1404/2810] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:00:39 2022] DELETE /identity/v3/users/af34317cec894fe4a96f4eec053f183a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:00:40.142442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:40.142838 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:40.156285 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:40.167669 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:00:40.169055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:40.247781 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1407/2811] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:00:40.261195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:40.263316 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:40.338356 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2276a1-33df-4775-af86-cc495218c865 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:40.343305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:40.343305 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:40.346547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:40.346547 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:40.360207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:40.361928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:40.372984 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1405/2812] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:00:41.116421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:00:41.118077 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:00:41.133155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a77b59-81c2-4f3f-a4cd-cb7d6f07e879 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:00:41.139774 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:00:41.141367 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:00:41.141872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:00:41.142601 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] PATH_INFO: `/v3/projects/ec5ad031468148f8947a0cbdb453e2c5` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:00:41.144938 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5ad031468148f8947a0cbdb453e2c5)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:00:41.147128 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:00:41.222532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5ad031468148f8947a0cbdb453e2c5', 'request_id': 'req-b9d66c9b-7295-426b-9330-ba2a432d9f88'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:41.234637 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5ad031468148f8947a0cbdb453e2c5', 'request_id': 'req-b9d66c9b-7295-426b-9330-ba2a432d9f88'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:41.243577 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5ad031468148f8947a0cbdb453e2c5', 'request_id': 'req-b9d66c9b-7295-426b-9330-ba2a432d9f88'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:41.258431 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] The token cache is being invalidate because project ec5ad031468148f8947a0cbdb453e2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:00:41.259693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:41.259693 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b9d66c9b-7295-426b-9330-ba2a432d9f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:00:41.261386 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1408/2813] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:00:41 2022] DELETE /identity/v3/projects/ec5ad031468148f8947a0cbdb453e2c5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:01:00.557257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:00.559430 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:00.580598 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6353c3b-851c-4cd2-9bc6-037f3b3a692d service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:00.641852 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:00.642634 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:00.642977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:00.643498 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:00.645539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:00.721354 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:00.722819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:00.733855 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1406/2814] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:01:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 15:01:01.901373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b82d6-231d-4575-bfca-4023fa320270 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:01.902355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b82d6-231d-4575-bfca-4023fa320270 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:01.902681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b82d6-231d-4575-bfca-4023fa320270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:01.938436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-907b82d6-231d-4575-bfca-4023fa320270 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:01.941083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-907b82d6-231d-4575-bfca-4023fa320270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:02.026926 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1409/2815] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:01:02.111287 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:02.114005 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:02.218416 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5f9dc-76d7-48d6-8f4b-24f7df875695 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:02.224193 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f953f-3884-4366-924f-31861d658062 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:02.224914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f953f-3884-4366-924f-31861d658062 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:02.225202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f953f-3884-4366-924f-31861d658062 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:02.225482 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f953f-3884-4366-924f-31861d658062 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:02.234986 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f953f-3884-4366-924f-31861d658062 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:02.236621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f953f-3884-4366-924f-31861d658062 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:02.246231 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1407/2816] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 15:01:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 04 15:01:02.973531 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f055b5-7bee-416a-a841-33cacb9771c4 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:02.974389 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f055b5-7bee-416a-a841-33cacb9771c4 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:02.974589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f055b5-7bee-416a-a841-33cacb9771c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:03.007189 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-45f055b5-7bee-416a-a841-33cacb9771c4 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:03.009197 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-45f055b5-7bee-416a-a841-33cacb9771c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:03.036541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1410/2817] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 4 15:01:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:03.071029 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f953f-3884-4366-924f-31861d658062 service swift] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:03.082881 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f953f-3884-4366-924f-31861d658062 service swift] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:03.098715 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:03.100180 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:03.100923 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:03.101533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:03.121735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:03.124712 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:03.143261 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1408/2818] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 15:01:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 15:01:03.568185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71538c17-100b-4a25-ba7b-7d705acf16a1 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:03.569087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71538c17-100b-4a25-ba7b-7d705acf16a1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:03.569369 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71538c17-100b-4a25-ba7b-7d705acf16a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:03.590455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-71538c17-100b-4a25-ba7b-7d705acf16a1 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:03.591888 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-71538c17-100b-4a25-ba7b-7d705acf16a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:03.599982 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1411/2819] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 15:01:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:01:03.617601 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:03.630325 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02d0c-f41d-45c7-b708-80d9c1e275c8 service swift] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:03.636228 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:03.637098 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:03.637448 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:03.637760 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:03.647429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:03.649224 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:03.659897 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1409/2820] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 15:01:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 15:01:04.166288 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:04.167080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:04.167080 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:04.186929 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:04.188179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:04.196378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1412/2821] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 4 15:01:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:01:04.216833 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:04.236464 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c75bc-04ab-4728-9c29-ea34ac053ff7 service swift] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:04.243320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ac07d-5fae-4636-a042-a4fc5198ea1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:04.244785 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac07d-5fae-4636-a042-a4fc5198ea1f service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:04.245177 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac07d-5fae-4636-a042-a4fc5198ea1f service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:04.245533 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac07d-5fae-4636-a042-a4fc5198ea1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:04.260218 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ac07d-5fae-4636-a042-a4fc5198ea1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:04.262442 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ac07d-5fae-4636-a042-a4fc5198ea1f service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:04.274486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1410/2822] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 15:01:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 15:01:06.880281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:06.881757 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:06.964087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea369a7-9a03-48c6-afd9-8165659aec20 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:06.969555 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:06.969924 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:06.970388 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:06.970824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] PATH_INFO: `/v3/users/b2609760306b4f0ca07caaa8b3568a3d` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:06.972242 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2609760306b4f0ca07caaa8b3568a3d)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:06.974483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:07.045549 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2609760306b4f0ca07caaa8b3568a3d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.066820 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2609760306b4f0ca07caaa8b3568a3d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.067980 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2609760306b4f0ca07caaa8b3568a3d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.068983 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2609760306b4f0ca07caaa8b3568a3d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.082198 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2609760306b4f0ca07caaa8b3568a3d'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.098575 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1413/2823] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:01:06 2022] DELETE /identity/v3/users/b2609760306b4f0ca07caaa8b3568a3d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:01:07.104739 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:07.105329 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:07.105442 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:07.130118 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:07.132427 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:07.218380 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1411/2824] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:07.239421 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:07.241008 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:07.320316 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472992-50cb-493b-889d-d23290bf9971 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:07.326749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c2392-4aec-4aeb-9824-8cb416e2eaf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:07.327922 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2392-4aec-4aeb-9824-8cb416e2eaf0 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:07.328446 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2392-4aec-4aeb-9824-8cb416e2eaf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:07.328840 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c2392-4aec-4aeb-9824-8cb416e2eaf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:07.340420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c2392-4aec-4aeb-9824-8cb416e2eaf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:07.342179 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c2392-4aec-4aeb-9824-8cb416e2eaf0 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:07.354126 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1414/2825] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:01:07.582137 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:07.584410 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:07.615016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260509d-d590-4a0a-97ee-08385bab3721 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:07.624207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:07.625822 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:07.626239 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:07.626624 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] PATH_INFO: `/v3/projects/063b0fdaa49a4cb6a7db0defa0e18f8d` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:07.630426 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063b0fdaa49a4cb6a7db0defa0e18f8d)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:07.633963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:07.687337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063b0fdaa49a4cb6a7db0defa0e18f8d', 'request_id': 'req-2c56fabf-e27b-44bf-b4ee-61d766688301'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.693957 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063b0fdaa49a4cb6a7db0defa0e18f8d', 'request_id': 'req-2c56fabf-e27b-44bf-b4ee-61d766688301'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.700588 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063b0fdaa49a4cb6a7db0defa0e18f8d', 'request_id': 'req-2c56fabf-e27b-44bf-b4ee-61d766688301'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.721070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] The token cache is being invalidate because project 063b0fdaa49a4cb6a7db0defa0e18f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:01:07.721419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.722391 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-2c56fabf-e27b-44bf-b4ee-61d766688301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:01:07.724945 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1412/2826] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:01:07 2022] DELETE /identity/v3/projects/063b0fdaa49a4cb6a7db0defa0e18f8d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:01:07.733681 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7c8cc-a9e3-4ae0-9cd5-ae6e527f292b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:07.734109 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7c8cc-a9e3-4ae0-9cd5-ae6e527f292b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:07.734540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7c8cc-a9e3-4ae0-9cd5-ae6e527f292b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:07.763535 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-86d7c8cc-a9e3-4ae0-9cd5-ae6e527f292b None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:07.765779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-86d7c8cc-a9e3-4ae0-9cd5-ae6e527f292b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:07.834632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1415/2827] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:07.844609 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533acb36-b7a4-4ca2-a758-1312c36ae509 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:07.845265 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533acb36-b7a4-4ca2-a758-1312c36ae509 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:07.845771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533acb36-b7a4-4ca2-a758-1312c36ae509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:07.871928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-533acb36-b7a4-4ca2-a758-1312c36ae509 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:07.873951 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-533acb36-b7a4-4ca2-a758-1312c36ae509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:07.884146 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1413/2828] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:07.889087 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7c8cc-a9e3-4ae0-9cd5-ae6e527f292b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:07.900894 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7c8cc-a9e3-4ae0-9cd5-ae6e527f292b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:07.906333 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:07.907054 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:07.907327 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:07.907629 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] PATH_INFO: `/v3/domains` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:07.908453 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91630) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 15:01:07.908765 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:07.910516 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:07.925061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1416/2829] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 15:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 04 15:01:07.929914 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533acb36-b7a4-4ca2-a758-1312c36ae509 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:07.940548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533acb36-b7a4-4ca2-a758-1312c36ae509 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:07.946095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:07.946805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:07.947091 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:07.947369 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:07.948104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:07.950181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:07.983202 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1414/2830] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:01:07 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:01:07.987967 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:08.003129 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f5e14-bc2a-417d-aa8b-6a2237405655 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:08.085331 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:08.086015 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:08.086309 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:08.086627 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:08.087543 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:08.089667 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:08.112939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1417/2831] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:01:07 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:01:08.117410 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:08.128909 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71ef94-3251-4a9c-91af-3cca3f9ad3dd admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:08.134435 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:08.135130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:08.135437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:08.135680 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:08.136649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:08.138347 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:08.150985 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1415/2832] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:01:08.156038 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:08.172044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc2db92-fae1-438c-8085-3e09a54f8df6 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:08.179830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3fc6e9-172a-4279-bf6a-817d66f29596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:08.180998 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fc6e9-172a-4279-bf6a-817d66f29596 admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:08.181464 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fc6e9-172a-4279-bf6a-817d66f29596 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:08.181897 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fc6e9-172a-4279-bf6a-817d66f29596 admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:08.183156 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fc6e9-172a-4279-bf6a-817d66f29596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:08.185907 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3fc6e9-172a-4279-bf6a-817d66f29596 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:08.201064 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1418/2833] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:01:08.205890 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:08.218024 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485901c5-430a-41f6-bb92-5ba3954f87ea admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:08.223931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:08.224868 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] REQUEST_METHOD: `PUT` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:08.225148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:08.225419 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] PATH_INFO: `/v3/projects/2036e616b4c8436aa631073699f30a33/users/a970ff60373e4432ae62db30b528c17b/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:08.242272 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2036e616b4c8436aa631073699f30a33, user_id=a970ff60373e4432ae62db30b528c17b, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:08.245173 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:08.261629 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1416/2834] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:01:08 2022] PUT /identity/v3/projects/2036e616b4c8436aa631073699f30a33/users/a970ff60373e4432ae62db30b528c17b/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:01:08.267936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897550-b004-41da-889b-295c667475d7 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:08.268413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897550-b004-41da-889b-295c667475d7 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:08.268940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897550-b004-41da-889b-295c667475d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:08.288747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-f3897550-b004-41da-889b-295c667475d7 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:08.290119 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f3897550-b004-41da-889b-295c667475d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:08.381747 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1419/2835] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:08.392130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:08.394148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:08.463053 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ad6c0-9b55-4e94-a35b-fd829e23597a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:08.466767 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:08.467440 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:08.467718 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:08.467990 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:08.478195 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:08.479977 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:08.495886 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1417/2836] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:01:13.891400 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673b8f3-8e20-473d-8f4c-b411aea30993 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:13.892061 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673b8f3-8e20-473d-8f4c-b411aea30993 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:13.892349 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673b8f3-8e20-473d-8f4c-b411aea30993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:13.952368 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-7673b8f3-8e20-473d-8f4c-b411aea30993 None None] MFA Rules not processed for user `a970ff60373e4432ae62db30b528c17b`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:13.955741 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-7673b8f3-8e20-473d-8f4c-b411aea30993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:14.071355 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1420/2837] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:14.076411 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:14.086861 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9009c0-46aa-4bb1-b6b1-783acc4c6813 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:14.092080 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:14.092764 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:14.093123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:14.093399 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] PATH_INFO: `/v3/projects` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:14.094381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:14.096331 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:14.134368 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1418/2838] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:01:14 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 15:01:14.135592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673b8f3-8e20-473d-8f4c-b411aea30993 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:14.147881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673b8f3-8e20-473d-8f4c-b411aea30993 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:14.243096 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:14.244186 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:14.244652 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:14.245151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] PATH_INFO: `/v3/users` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:14.246170 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:14.248659 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:14.287092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1421/2839] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:01:14 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 15:01:14.289027 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:14.318828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b904a-4100-4559-8bec-9a68dbd0ba53 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:14.328548 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:14.329541 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:14.330012 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:14.330494 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:14.331815 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:14.333737 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:14.351811 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1419/2840] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:01:14.357476 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:14.370412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4439d4-6d13-464e-aaf8-10b7d1ebd203 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:14.378580 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:14.379832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:14.380299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:14.380683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] PATH_INFO: `/v3/projects/1643544cd6684d30af2abd8a8800caf3/users/4d9b871cda6e41b28b6d3ecf0fd09c3e/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:14.401743 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1643544cd6684d30af2abd8a8800caf3, user_id=4d9b871cda6e41b28b6d3ecf0fd09c3e, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:14.406423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:14.425468 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1422/2841] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:01:14 2022] PUT /identity/v3/projects/1643544cd6684d30af2abd8a8800caf3/users/4d9b871cda6e41b28b6d3ecf0fd09c3e/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:01:14.431148 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:14.448564 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dadcd-6bba-4783-b215-892d1255199a admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:14.528262 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75103ac-ca23-4b22-9398-ba9a436674d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:14.529500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75103ac-ca23-4b22-9398-ba9a436674d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:14.529996 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75103ac-ca23-4b22-9398-ba9a436674d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:14.530479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75103ac-ca23-4b22-9398-ba9a436674d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:14.531926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75103ac-ca23-4b22-9398-ba9a436674d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:14.534981 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75103ac-ca23-4b22-9398-ba9a436674d9 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:14.547766 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1420/2842] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 04 15:01:14.552546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:14.564456 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6033a-a2bc-443a-b199-8569a97d041a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:14.570146 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5c16a-58e5-41e9-9ac8-1f3dd483389b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:14.571145 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c16a-58e5-41e9-9ac8-1f3dd483389b admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:14.571424 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c16a-58e5-41e9-9ac8-1f3dd483389b admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:14.571746 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c16a-58e5-41e9-9ac8-1f3dd483389b admin admin] PATH_INFO: `/v3/projects/1643544cd6684d30af2abd8a8800caf3/users/4d9b871cda6e41b28b6d3ecf0fd09c3e/roles/11ee08052a1446a2adaf40beef1e04c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:14.574125 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5c16a-58e5-41e9-9ac8-1f3dd483389b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1643544cd6684d30af2abd8a8800caf3, user_id=4d9b871cda6e41b28b6d3ecf0fd09c3e, role_id=11ee08052a1446a2adaf40beef1e04c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:14.577081 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5c16a-58e5-41e9-9ac8-1f3dd483389b admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:14.590372 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1423/2843] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:01:14 2022] PUT /identity/v3/projects/1643544cd6684d30af2abd8a8800caf3/users/4d9b871cda6e41b28b6d3ecf0fd09c3e/roles/11ee08052a1446a2adaf40beef1e04c3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:01:16.388465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:16.389096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:16.389233 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:16.413320 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] MFA Rules not processed for user `bc3501400c3b40f7993871f59e8571ff`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:16.415370 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:16.519583 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1421/2844] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:21.832663 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3164f-80bb-4b9a-b40a-4579c5439d1c None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:21.833824 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3164f-80bb-4b9a-b40a-4579c5439d1c None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:21.834277 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3164f-80bb-4b9a-b40a-4579c5439d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:21.880194 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-39d3164f-80bb-4b9a-b40a-4579c5439d1c None None] MFA Rules not processed for user `4d9b871cda6e41b28b6d3ecf0fd09c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:21.882624 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-39d3164f-80bb-4b9a-b40a-4579c5439d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:22.045068 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1424/2845] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:01:22.067622 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:22.072129 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:22.215882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4d5ee-5111-49f1-91bb-948c6e5bf17a None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:22.221008 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd3eb9e77dc64869b03b1190cf30e54b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:22.222022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:22.222500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:22.222968 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:22.345830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:22.348278 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:22.367036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1422/2846] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:01:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:01:22.509299 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437e693-ef42-43de-877d-8c98aaa071a1 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:22.509606 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437e693-ef42-43de-877d-8c98aaa071a1 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:22.509859 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437e693-ef42-43de-877d-8c98aaa071a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:22.535735 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-e437e693-ef42-43de-877d-8c98aaa071a1 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:22.537455 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-e437e693-ef42-43de-877d-8c98aaa071a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:22.624060 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1425/2847] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:22.638071 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:22.640079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:22.760066 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8013e9-bef9-4d4b-9c5b-bafd9a6eb757 service glance] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:22.768989 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233f2e4-abcd-4e38-a88d-a8933cff02c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:22.769958 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233f2e4-abcd-4e38-a88d-a8933cff02c1 service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:22.770337 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233f2e4-abcd-4e38-a88d-a8933cff02c1 service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:22.770775 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233f2e4-abcd-4e38-a88d-a8933cff02c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:22.784858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233f2e4-abcd-4e38-a88d-a8933cff02c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:22.786649 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9233f2e4-abcd-4e38-a88d-a8933cff02c1 service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:22.798716 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1423/2848] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 15:01:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 15:01:33.408583 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc46ba-c964-44ba-aedd-85714ea58ff0 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:33.409157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc46ba-c964-44ba-aedd-85714ea58ff0 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:33.409157 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc46ba-c964-44ba-aedd-85714ea58ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:33.428524 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-08cc46ba-c964-44ba-aedd-85714ea58ff0 None None] MFA Rules not processed for user `a970ff60373e4432ae62db30b528c17b`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:33.429835 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-08cc46ba-c964-44ba-aedd-85714ea58ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:33.443571 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1426/2849] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:01:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:35.343574 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:35.344062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:35.344145 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:35.362682 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:01:35.363916 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:01:35.372732 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1424/2850] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 4 15:01:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:01:35.386683 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc46ba-c964-44ba-aedd-85714ea58ff0 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:01:35.399459 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc46ba-c964-44ba-aedd-85714ea58ff0 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:01:35.404376 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:01:35.405044 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:01:35.405321 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:01:35.405589 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:01:35.414609 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:01:35.416294 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:01:35.425900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1427/2851] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 15:01:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 15:02:09.395511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:09.397653 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:09.458126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b01bb-d559-4b93-830f-fcca7c20e4b0 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:09.461943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:09.462617 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:09.462896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:09.463168 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:09.472783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:09.474237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:09.483937 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1425/2852] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:02:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:02:21.129458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:21.138026 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eebf5-d3d1-4863-86bf-833c4b32dfbd service swift] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:21.142155 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:21.142821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:21.143410 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:21.143752 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:21.145797 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:21.226416 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:21.227832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:21.239844 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1428/2853] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:02:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:02:22.418607 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:22.427891 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a38180-7de7-49a4-b620-297bbf9ad7a8 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:22.432089 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:22.433095 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:22.433465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:22.433738 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:22.435736 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:22.451064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:22.452501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:22.464116 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1426/2854] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:02:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:02:26.655211 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:26.665051 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8076a4-d6e3-4e1e-b839-d1832dc4288f service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:26.668784 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:26.669472 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:26.669756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:26.670034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:26.671934 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:26.736485 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:26.738041 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:26.748244 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1429/2855] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:02:27.483873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:27.485535 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:27.500021 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d0f1-9517-49f6-9669-6b093b3dd1ea service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:27.505122 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:27.505967 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:27.506248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:27.506522 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] PATH_INFO: `/v3/users/bc3501400c3b40f7993871f59e8571ff` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:27.507784 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3501400c3b40f7993871f59e8571ff)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:27.509771 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:27.559755 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3501400c3b40f7993871f59e8571ff'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.560433 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3501400c3b40f7993871f59e8571ff'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.561076 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3501400c3b40f7993871f59e8571ff'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.567974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3501400c3b40f7993871f59e8571ff'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.579696 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-45d202da-07f5-4975-8cff-fa1295dc1853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3501400c3b40f7993871f59e8571ff'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.592858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1427/2856] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:02:27 2022] DELETE /identity/v3/users/bc3501400c3b40f7993871f59e8571ff => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 15:02:27.770981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:27.783581 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168d4a-ab7e-44fe-abf0-dcc0f7e55d6a service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:27.843544 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:27.844536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:27.844912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:27.845264 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] PATH_INFO: `/v3/projects/0a60c17ae45844c490bcefc6bc54c879` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:27.846536 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a60c17ae45844c490bcefc6bc54c879)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:27.848484 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:27.899137 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a60c17ae45844c490bcefc6bc54c879', 'request_id': 'req-dc72d196-582c-4c4b-ac9f-2263ff9c99da'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.906658 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a60c17ae45844c490bcefc6bc54c879', 'request_id': 'req-dc72d196-582c-4c4b-ac9f-2263ff9c99da'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.912392 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a60c17ae45844c490bcefc6bc54c879', 'request_id': 'req-dc72d196-582c-4c4b-ac9f-2263ff9c99da'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.924981 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] The token cache is being invalidate because project 0a60c17ae45844c490bcefc6bc54c879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:02:27.925233 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.925954 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-dc72d196-582c-4c4b-ac9f-2263ff9c99da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:27.927804 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1430/2857] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:02:27 2022] DELETE /identity/v3/projects/0a60c17ae45844c490bcefc6bc54c879 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:02:27.933172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e747ca-5504-48e5-9164-ca5f2f01c18f None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:27.933172 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e747ca-5504-48e5-9164-ca5f2f01c18f None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:27.933493 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e747ca-5504-48e5-9164-ca5f2f01c18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:27.951659 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-07e747ca-5504-48e5-9164-ca5f2f01c18f None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:02:27.953477 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-07e747ca-5504-48e5-9164-ca5f2f01c18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:28.025540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1428/2858] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:02:28.027573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0d8f3-e39f-47d2-8e6c-6831a3c54563 None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.028175 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0d8f3-e39f-47d2-8e6c-6831a3c54563 None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.028668 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0d8f3-e39f-47d2-8e6c-6831a3c54563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.047585 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-50b0d8f3-e39f-47d2-8e6c-6831a3c54563 None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:02:28.048958 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-50b0d8f3-e39f-47d2-8e6c-6831a3c54563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:28.058759 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1431/2859] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:02:28.063237 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e747ca-5504-48e5-9164-ca5f2f01c18f None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:28.073300 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e747ca-5504-48e5-9164-ca5f2f01c18f None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:28.078409 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:28.079151 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.079497 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.079832 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] PATH_INFO: `/v3/domains` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.080647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 04 15:02:28.081055 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:28.082726 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:28.099830 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1429/2860] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 4 15:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 04 15:02:28.101363 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0d8f3-e39f-47d2-8e6c-6831a3c54563 None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:28.110690 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0d8f3-e39f-47d2-8e6c-6831a3c54563 None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:28.115829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:28.116501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.116963 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.117379 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.118083 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:28.120029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:28.152390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1432/2861] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 4 15:02:28 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 04 15:02:28.153600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:28.163860 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e41f7-cfad-436a-bb5f-f514c5419487 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:28.221648 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:28.222546 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.222824 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.223100 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] PATH_INFO: `/v3/users` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.223806 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:28.226062 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:28.247037 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1430/2862] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 4 15:02:28 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 04 15:02:28.252094 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:28.263141 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f488b-5425-483a-a8f1-edc1454a1bbf admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:28.267893 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:28.268541 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.268855 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.269131 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.269940 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:28.271718 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:28.283334 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1433/2863] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:02:28.287345 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:28.297389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69f4e6-81a2-4290-bdab-f6ae50bbf6ed admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:28.302203 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629e1b2-d069-4bc3-83fd-e483c2820349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:28.302899 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629e1b2-d069-4bc3-83fd-e483c2820349 admin admin] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.303257 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629e1b2-d069-4bc3-83fd-e483c2820349 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.303660 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629e1b2-d069-4bc3-83fd-e483c2820349 admin admin] PATH_INFO: `/v3/roles` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.304454 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629e1b2-d069-4bc3-83fd-e483c2820349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:28.306117 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629e1b2-d069-4bc3-83fd-e483c2820349 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:28.319211 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1431/2864] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 4 15:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 04 15:02:28.323839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:28.334548 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d9dc-dba5-4244-a74c-709da54ae0cc admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:28.339701 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:28.340612 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.340931 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.341208 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] PATH_INFO: `/v3/projects/cb9914eb44624bb8a0ce8cf60ea6fcc2/users/14a70b899ebb494688e5c8a99838f95f/roles/4e8a255633994167b69cee0662997d5e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.356730 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9914eb44624bb8a0ce8cf60ea6fcc2, user_id=14a70b899ebb494688e5c8a99838f95f, role_id=4e8a255633994167b69cee0662997d5e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:28.359483 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:28.374049 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1434/2865] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 4 15:02:28 2022] PUT /identity/v3/projects/cb9914eb44624bb8a0ce8cf60ea6fcc2/users/14a70b899ebb494688e5c8a99838f95f/roles/4e8a255633994167b69cee0662997d5e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:02:28.385175 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] REQUEST_METHOD: `POST` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.385465 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.385742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.412003 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.auth.core [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] MFA Rules not processed for user `14a70b899ebb494688e5c8a99838f95f`. Rule list: `[]` (Enabled: `True`). {{(pid=91629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:02:28.413251 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:28.489805 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1432/2866] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 04 15:02:28.503501 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:28.505632 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:28.572378 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3724fff-2ce7-40b3-9d0c-9085ca6a9b56 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:28.577200 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4aed1720aea4ffa975f7936a1bdb3c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:28.577881 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:28.578151 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:28.578420 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:28.588962 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:28.590436 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:28.600229 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1435/2867] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:02:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:02:33.617398 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:33.619551 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:33.717463 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e0e73-9532-4709-9c85-2d24155fd849 None None] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:33.722248 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:33.723143 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:33.723444 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:33.723714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] PATH_INFO: `/v3/users/3075413bbcba4104bb5fbb907ab2558a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:33.724962 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3075413bbcba4104bb5fbb907ab2558a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:33.726964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:33.776511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3075413bbcba4104bb5fbb907ab2558a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:33.777301 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3075413bbcba4104bb5fbb907ab2558a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:33.777938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3075413bbcba4104bb5fbb907ab2558a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:33.787511 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3075413bbcba4104bb5fbb907ab2558a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:33.800260 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3075413bbcba4104bb5fbb907ab2558a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:33.815296 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1433/2868] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:02:33 2022] DELETE /identity/v3/users/3075413bbcba4104bb5fbb907ab2558a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:02:34.006636 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:34.019509 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c93b1-63ca-4d77-9ae5-cad9027bb1ff service cinder] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:34.080822 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:34.081716 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:34.082009 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:34.082281 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] PATH_INFO: `/v3/projects/da7a630cbd2f476a81d60db712b4dbd3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:34.083654 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7a630cbd2f476a81d60db712b4dbd3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:34.085814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:34.138022 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7a630cbd2f476a81d60db712b4dbd3', 'request_id': 'req-5be6c710-5c3a-42a9-96b1-7250fc267b5a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.148067 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7a630cbd2f476a81d60db712b4dbd3', 'request_id': 'req-5be6c710-5c3a-42a9-96b1-7250fc267b5a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.153936 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7a630cbd2f476a81d60db712b4dbd3', 'request_id': 'req-5be6c710-5c3a-42a9-96b1-7250fc267b5a'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.165724 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] The token cache is being invalidate because project da7a630cbd2f476a81d60db712b4dbd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:02:34.166004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.166723 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.168661 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1436/2869] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:02:34 2022] DELETE /identity/v3/projects/da7a630cbd2f476a81d60db712b4dbd3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:02:34.172864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:34.174395 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:34.193523 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec010bb1-a7d1-46c7-9f78-d8f6806eb022 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:34.262096 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:34.262987 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:34.263267 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:34.263539 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] PATH_INFO: `/v3/users/cc91883c9d15451fabe899531d318a1f` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:34.264913 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc91883c9d15451fabe899531d318a1f)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:34.266848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:34.322090 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc91883c9d15451fabe899531d318a1f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.322789 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc91883c9d15451fabe899531d318a1f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.323621 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc91883c9d15451fabe899531d318a1f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.333759 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc91883c9d15451fabe899531d318a1f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.351781 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc91883c9d15451fabe899531d318a1f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.366926 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1434/2870] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:02:34 2022] DELETE /identity/v3/users/cc91883c9d15451fabe899531d318a1f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:02:34.561488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:34.563488 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:34.581423 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6c710-5c3a-42a9-96b1-7250fc267b5a admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:34.648679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:34.649592 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:34.649872 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:34.650149 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] PATH_INFO: `/v3/projects/ed26aba538f0415fa3fe812554c97365` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:34.651390 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed26aba538f0415fa3fe812554c97365)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:34.653341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:34.702697 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed26aba538f0415fa3fe812554c97365', 'request_id': 'req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.711282 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed26aba538f0415fa3fe812554c97365', 'request_id': 'req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.717143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed26aba538f0415fa3fe812554c97365', 'request_id': 'req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.730092 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] The token cache is being invalidate because project ed26aba538f0415fa3fe812554c97365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:02:34.730469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.731341 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-e48ce4a6-4998-419d-9179-de2ae1fe6c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:34.733462 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1437/2871] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:02:34 2022] DELETE /identity/v3/projects/ed26aba538f0415fa3fe812554c97365 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 04 15:02:36.755851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:36.757563 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:36.771252 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92c285-e1f8-4741-beff-dfce4acb98aa admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:36.828165 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:36.829079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:36.829362 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:36.829611 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] PATH_INFO: `/v3/users/14a70b899ebb494688e5c8a99838f95f` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:36.830873 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a70b899ebb494688e5c8a99838f95f)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:36.832783 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:36.883557 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a70b899ebb494688e5c8a99838f95f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:36.884208 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a70b899ebb494688e5c8a99838f95f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:36.884828 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a70b899ebb494688e5c8a99838f95f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:36.891896 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a70b899ebb494688e5c8a99838f95f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:36.904710 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a70b899ebb494688e5c8a99838f95f'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:36.918964 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1435/2872] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:02:36 2022] DELETE /identity/v3/users/14a70b899ebb494688e5c8a99838f95f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:02:36.923942 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:36.924412 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:36.924540 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:36.944527 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] MFA Rules not processed for user `f6614ed1ab4445508051598d04c940bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:02:36.945796 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:37.014912 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1438/2873] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 4 15:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:02:37.021524 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:37.023059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:37.053602 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:37.055176 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:37.089594 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548d030-9954-4190-9421-2fa4d08c3317 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:37.093748 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:37.094450 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:37.094735 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:37.095026 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:37.104961 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:37.106402 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:37.116974 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1436/2874] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:02:37.118686 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb083fb-bb75-436a-9cc4-d92614ba05de None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:37.122550 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:37.123250 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:37.123532 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:37.123867 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:37.126034 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:37.136357 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:37.137871 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:37.146055 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1439/2875] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:02:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 04 15:02:37.558504 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:02:37.560109 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:02:37.573769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec55d24-8200-4cb4-8a0d-6341cb38eac7 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:02:37.578647 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:02:37.580429 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:02:37.581104 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:02:37.581381 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] PATH_INFO: `/v3/projects/cb9914eb44624bb8a0ce8cf60ea6fcc2` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:02:37.582920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9914eb44624bb8a0ce8cf60ea6fcc2)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:02:37.584532 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:02:37.645848 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9914eb44624bb8a0ce8cf60ea6fcc2', 'request_id': 'req-ba031376-bb8e-45cb-8f01-8386fcfb487d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:37.652113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9914eb44624bb8a0ce8cf60ea6fcc2', 'request_id': 'req-ba031376-bb8e-45cb-8f01-8386fcfb487d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:37.657769 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9914eb44624bb8a0ce8cf60ea6fcc2', 'request_id': 'req-ba031376-bb8e-45cb-8f01-8386fcfb487d'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:37.671580 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] The token cache is being invalidate because project cb9914eb44624bb8a0ce8cf60ea6fcc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:02:37.671867 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:37.672519 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:02:37.674119 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1437/2876] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:02:37 2022] DELETE /identity/v3/projects/cb9914eb44624bb8a0ce8cf60ea6fcc2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:03:14.569027 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:14.577821 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e029b-2b1a-450e-a9ec-e1efb75ee241 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:14.628469 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:14.629164 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:14.629442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:14.629722 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:14.631587 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:14.641648 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:14.643063 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:14.651842 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1440/2877] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:03:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:03:14.728127 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:14.729690 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:14.742486 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba031376-bb8e-45cb-8f01-8386fcfb487d admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:14.747153 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:14.747839 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:14.748131 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:14.748754 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:14.750982 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:14.831392 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:14.832782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:14.844880 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1438/2878] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:03:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:03:14.923029 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:14.924695 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:14.998939 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d97767-d0ac-4af6-a182-1f2accc1c586 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:15.008779 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac21bca8da642338571f821ef5ea121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:15.009471 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:15.009754 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:15.010024 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:15.012418 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:15.025671 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:15.027259 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:15.043909 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1441/2879] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:03:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:03:16.595934 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:16.597424 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:16.611883 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5823445-caf2-4e9c-8d66-2208906b4530 service neutron] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:16.617807 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:16.618479 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:16.618742 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:16.619015 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:16.621078 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:16.636437 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:16.637704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:16.650047 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1439/2880] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:03:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:03:22.081557 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:22.103610 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800ede6-b1b5-4359-a7d4-1375416dcd01 service placement] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:22.117017 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2bb7e-6a53-4463-844f-953be98d528d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:22.118373 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2bb7e-6a53-4463-844f-953be98d528d service nova] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:22.118900 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2bb7e-6a53-4463-844f-953be98d528d service nova] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:22.119449 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2bb7e-6a53-4463-844f-953be98d528d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:22.121857 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-09f2bb7e-6a53-4463-844f-953be98d528d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:22.143800 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2bb7e-6a53-4463-844f-953be98d528d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:22.145611 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2bb7e-6a53-4463-844f-953be98d528d service nova] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:22.158605 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1442/2881] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:03:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:03:29.199247 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:29.211123 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff61904-1587-4bd5-b3a6-29e308d4eb9b service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:29.217207 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '753f2914e5bc4e298b99529e230d9ee3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:29.218176 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:29.218777 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:29.219022 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:29.221501 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:29.235540 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:29.237389 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:29.249036 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1440/2882] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:03:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:03:30.646442 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] REQUEST_METHOD: `POST` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:30.647005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:30.647005 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:30.665673 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.auth.core [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] MFA Rules not processed for user `b32490f3e59845f0a2720ca879664b21`. Rule list: `[]` (Enabled: `True`). {{(pid=91630) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 04 15:03:30.667090 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:30.735987 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1443/2883] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 4 15:03:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 04 15:03:30.786404 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:30.787931 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:30.861932 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa376b-e603-4180-a4dd-926fc3918019 service nova] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:30.867882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4ed1c88f3cd478aafc83f4b76d67f7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:30.868543 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:30.868856 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:30.869135 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:30.878063 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:30.879668 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:30.887819 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1441/2884] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 4 15:03:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 04 15:03:38.003479 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:38.004974 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:38.079435 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb94cbb-0442-4b14-990c-7bb3d2c7a36b None None] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:38.084832 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:38.085714 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:38.085960 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:38.086260 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] PATH_INFO: `/v3/users/a970ff60373e4432ae62db30b528c17b` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:38.087521 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a970ff60373e4432ae62db30b528c17b)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:38.089500 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:38.148243 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a970ff60373e4432ae62db30b528c17b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.161158 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a970ff60373e4432ae62db30b528c17b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.161906 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a970ff60373e4432ae62db30b528c17b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.162573 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a970ff60373e4432ae62db30b528c17b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.171830 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a970ff60373e4432ae62db30b528c17b'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.184767 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1444/2885] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:03:38 2022] DELETE /identity/v3/users/a970ff60373e4432ae62db30b528c17b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:03:38.377606 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:38.388943 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94109697-4a7e-485f-9e6d-48ad3e21b17e service swift] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:38.445956 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:38.446851 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:38.447126 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:38.447400 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] PATH_INFO: `/v3/projects/2036e616b4c8436aa631073699f30a33` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:38.448600 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2036e616b4c8436aa631073699f30a33)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:38.450537 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:38.506733 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2036e616b4c8436aa631073699f30a33', 'request_id': 'req-e3f7ebc1-b176-432a-ae24-bab6ec518401'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.513620 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2036e616b4c8436aa631073699f30a33', 'request_id': 'req-e3f7ebc1-b176-432a-ae24-bab6ec518401'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.520158 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2036e616b4c8436aa631073699f30a33', 'request_id': 'req-e3f7ebc1-b176-432a-ae24-bab6ec518401'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.537475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] The token cache is being invalidate because project 2036e616b4c8436aa631073699f30a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:03:38.537786 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.538646 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.541016 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1442/2886] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:03:38 2022] DELETE /identity/v3/projects/2036e616b4c8436aa631073699f30a33 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:03:38.546252 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:38.548512 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:38.562538 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94439861-ed01-41fd-8fa9-54dd70e13739 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:38.629072 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:38.629978 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:38.630255 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:38.630564 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] PATH_INFO: `/v3/users/4d9b871cda6e41b28b6d3ecf0fd09c3e` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:38.631839 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9b871cda6e41b28b6d3ecf0fd09c3e)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:38.633834 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:38.683127 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9b871cda6e41b28b6d3ecf0fd09c3e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.695185 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9b871cda6e41b28b6d3ecf0fd09c3e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.695413 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9b871cda6e41b28b6d3ecf0fd09c3e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.696010 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9b871cda6e41b28b6d3ecf0fd09c3e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.704487 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9b871cda6e41b28b6d3ecf0fd09c3e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:38.716582 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1445/2887] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:03:38 2022] DELETE /identity/v3/users/4d9b871cda6e41b28b6d3ecf0fd09c3e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:03:38.905529 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:03:38.907144 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:03:38.920708 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7ebc1-b176-432a-ae24-bab6ec518401 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:03:38.979079 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:03:38.980059 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:03:38.980322 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:03:38.980592 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] PATH_INFO: `/v3/projects/1643544cd6684d30af2abd8a8800caf3` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:03:38.981818 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1643544cd6684d30af2abd8a8800caf3)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:03:38.983704 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:03:39.032333 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1643544cd6684d30af2abd8a8800caf3', 'request_id': 'req-0700ad77-0884-4da5-a640-7f2cb6435ad1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:39.038673 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1643544cd6684d30af2abd8a8800caf3', 'request_id': 'req-0700ad77-0884-4da5-a640-7f2cb6435ad1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:39.044133 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1643544cd6684d30af2abd8a8800caf3', 'request_id': 'req-0700ad77-0884-4da5-a640-7f2cb6435ad1'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:39.058864 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] The token cache is being invalidate because project 1643544cd6684d30af2abd8a8800caf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:03:39.059130 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:39.059842 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:03:39.061470 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1443/2888] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:03:38 2022] DELETE /identity/v3/projects/1643544cd6684d30af2abd8a8800caf3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:04:44.760110 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:04:44.761617 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:04:44.883709 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77690c-5f9c-45cb-8ccb-73229e27f809 admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:04:44.938598 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d678d5ee7d74896bbc1316b32ec81ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:04:44.939246 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] REQUEST_METHOD: `GET` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:04:44.939537 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:04:44.939814 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:04:44.941829 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:04:45.058679 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:04:45.060130 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:04:45.080510 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1446/2889] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 04 15:04:51.764204 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:04:51.766906 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:04:51.842188 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700ad77-0884-4da5-a640-7f2cb6435ad1 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:04:51.847858 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:04:51.848722 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:04:51.849064 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:04:51.849406 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] PATH_INFO: `/v3/users/f322d73b5c5d41d69e197a767d7aea75` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:04:51.851610 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f322d73b5c5d41d69e197a767d7aea75)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:04:51.854181 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:04:51.905349 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f322d73b5c5d41d69e197a767d7aea75'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:51.906070 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f322d73b5c5d41d69e197a767d7aea75'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:51.906776 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f322d73b5c5d41d69e197a767d7aea75'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:51.913928 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f322d73b5c5d41d69e197a767d7aea75'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:51.926500 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f322d73b5c5d41d69e197a767d7aea75'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:51.940727 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1444/2890] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:04:51 2022] DELETE /identity/v3/users/f322d73b5c5d41d69e197a767d7aea75 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:04:52.126889 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:04:52.136682 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b3640-af4f-48a6-b775-82939c6e2815 service neutron] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:04:52.193226 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:04:52.194004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:04:52.194280 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:04:52.194554 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] PATH_INFO: `/v3/projects/35023f0e4a0842778e3125cb024562c3` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:04:52.195749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] RBAC: Authorizing `identity:delete_project(project_id=35023f0e4a0842778e3125cb024562c3)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:04:52.197756 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:04:52.247809 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35023f0e4a0842778e3125cb024562c3', 'request_id': 'req-b2a5984e-9d78-493e-8682-b42622d7769e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.255505 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35023f0e4a0842778e3125cb024562c3', 'request_id': 'req-b2a5984e-9d78-493e-8682-b42622d7769e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.261385 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35023f0e4a0842778e3125cb024562c3', 'request_id': 'req-b2a5984e-9d78-493e-8682-b42622d7769e'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.272749 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] The token cache is being invalidate because project 35023f0e4a0842778e3125cb024562c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:04:52.273069 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.273928 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.275739 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1447/2891] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:04:52 2022] DELETE /identity/v3/projects/35023f0e4a0842778e3125cb024562c3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:04:52.279938 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:04:52.281475 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:04:52.294663 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29702150-3d58-42ef-a55f-20aae817e251 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:04:52.355235 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:04:52.356113 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:04:52.356394 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:04:52.356665 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] PATH_INFO: `/v3/users/36e88925b2674bdfa2bbfe8b04bd759a` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:04:52.357882 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e88925b2674bdfa2bbfe8b04bd759a)` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:04:52.359782 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:04:52.409628 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e88925b2674bdfa2bbfe8b04bd759a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.410289 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e88925b2674bdfa2bbfe8b04bd759a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.410901 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e88925b2674bdfa2bbfe8b04bd759a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.417920 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e88925b2674bdfa2bbfe8b04bd759a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.430567 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.notifications [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e88925b2674bdfa2bbfe8b04bd759a'} {{(pid=91629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.447591 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1445/2892] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 4 15:04:52 2022] DELETE /identity/v3/users/36e88925b2674bdfa2bbfe8b04bd759a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:04:52.624143 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Authenticating user token {{(pid=91630) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:04:52.625733 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.fernet_utils [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91630) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:04:52.639352 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5984e-9d78-493e-8682-b42622d7769e admin admin] Validating token access rules against request {{(pid=91630) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:04:52.698223 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6614ed1ab4445508051598d04c940bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62c598d6329d4180a1bbd38c5500b8cc', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91630) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:04:52.699004 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:04:52.699279 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:04:52.699546 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] PATH_INFO: `/v3/projects/b7a7cbefee9640cda9550c3dff2991ac` {{(pid=91630) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:04:52.700762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a7cbefee9640cda9550c3dff2991ac)` {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:04:52.702664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] RBAC: Authorization granted {{(pid=91630) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:04:52.766118 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a7cbefee9640cda9550c3dff2991ac', 'request_id': 'req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.776457 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a7cbefee9640cda9550c3dff2991ac', 'request_id': 'req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.782762 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a7cbefee9640cda9550c3dff2991ac', 'request_id': 'req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb'} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.794178 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] The token cache is being invalidate because project b7a7cbefee9640cda9550c3dff2991ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91630) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 04 15:04:52.794458 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.795088 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: DEBUG keystone.notifications [None req-a537416a-bd7e-4846-b31a-8ca5c3ace3eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91630) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 04 15:04:52.796664 dv-cinder-840493-1-iscsi devstack@keystone.service[91630]: [pid: 91630|app: 0|req: 1448/2893] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 4 15:04:52 2022] DELETE /identity/v3/projects/b7a7cbefee9640cda9550c3dff2991ac => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 04 15:05:22.692164 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Authenticating user token {{(pid=91629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 04 15:05:22.693778 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:05:22.790388 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d13d7-b238-4070-ad86-edc8e9b32f70 admin admin] Validating token access rules against request {{(pid=91629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 04 15:05:22.851310 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4f0fa-0c13-4625-990e-41076c3e4aff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ac21bca8da642338571f821ef5ea121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79beb601016d422faea4e6a5753cad40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 04 15:05:22.851963 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4f0fa-0c13-4625-990e-41076c3e4aff service placement] REQUEST_METHOD: `GET` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 04 15:05:22.852303 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4f0fa-0c13-4625-990e-41076c3e4aff service placement] SCRIPT_NAME: `/identity` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 04 15:05:22.852586 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4f0fa-0c13-4625-990e-41076c3e4aff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 04 15:05:22.854714 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.fernet_utils [None req-bea4f0fa-0c13-4625-990e-41076c3e4aff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 04 15:05:22.866408 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4f0fa-0c13-4625-990e-41076c3e4aff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 04 15:05:22.868069 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4f0fa-0c13-4625-990e-41076c3e4aff service placement] RBAC: Authorization granted {{(pid=91629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 04 15:05:22.881373 dv-cinder-840493-1-iscsi devstack@keystone.service[91629]: [pid: 91629|app: 0|req: 1446/2894] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 4 15:05:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)